Alpha
Contact
Use · LLM-jp-4.1 thinking

Is it good enough?

Ownership levelPartialnone·limited·partial·substantial·fullAnalytical input D ยท 54.4/100

This page is a projection of the one entry record, the Reliability factor that Use covers. The full verdict is set by all four factors together, floor-weighted so the weakest caps the whole.

Which domain expands which factor
  • AssessUse & modify + Transparency
  • ImplementData control + Reliability
  • UseReliability
  • SupportTransparency

Capabilities & modalities

A text-only reasoning ("thinking") model series for Japanese and English chat, mathematics, science question answering, code, instruction following, translation between Japanese and English, and, new in 4.1, tool calling (ev-card-41, ev-blog). Reasoning effort is set to low, medium or high in the chat template. The 4.1 post-training targeted STEM gains and shorter answers (ev-blog).

Sizes: 8B dense, 32B-A3B mixture of experts (3.8B active, 128 experts, 8 per token) and 33B dense. The vision-language model is a separate release and is not covered here.

Strength: Japanese-language tasks, where the 32B-A3B matched Qwen3-8B and gpt-oss-20b in the technical report. Weaker: English averages, GPQA Diamond and AIME against those peers (ev-techreport, ev-blog).

Context window & long-context behaviour

The configuration sets max_position_embeddings to 65,536 for all three sizes (ev-hub-api). The technical report describes extending context from 4K to 64K in mid-training through a 16K stage (ev-techreport). The publisher's blog reports that on BFCL's multi-turn long-context category the 33B model beats the 8B and 32B-A3B models by the widest margin, and that this is also the category where LLM-jp-4.1 trails Qwen3.5 most (ev-blog). No needle-in-a-haystack or long-context reasoning benchmark is published; test your own long inputs.

Prompt format & chat template

The chat template is a modified version of the OpenAI Harmony response format, with analysis (reasoning), commentary (tool calls) and final channels. The 4.1 changes: parallel tool calls end all but the last call with <|end|>, and reasoning is handled differently across turns from standard Harmony (ev-blog, ev-card-41).

Rules from the publisher (ev-cookbook, ev-card-41):

  • load the tokenizer bundled with the model; the openai-harmony library's tokenizer has different token IDs;
  • use apply_chat_template with add_generation_prompt=True and pass reasoning_effort;
  • parse token IDs, not decoded text, when message boundaries matter, because user text resembling special tokens such as <|...|> is ambiguous;
  • the 4.0 GGUF templates lack the declaration that selects the 4.1 parser, so use 4.1 GGUF files.

Language coverage

The model cards tag English and Japanese. Pre-training also uses Chinese and Korean web data (Chinese and Korean FineWeb-2 portions and Wikipedia) and code, with the tokenizer trained on English and code 0.68, Japanese 0.18, Chinese 0.09 and Korean 0.05 (ev-techreport, ev-corpus-v4). Post-training data is Japanese and English plus a small Korean maths subset (ev-sft-dpo). Chinese and Korean output quality and safety are not evaluated by the publisher, so treat them as unverified.

Japanese-specific work: Japanese Wikipedia, e-Gov legal texts, patents, Diet minutes, KAKEN abstracts, Aozora Bunko books and National Diet Library web archive are in the pre-training corpus (ev-corpus-v4); the post-training set includes a Japanese maths dataset aligned with the Ministry of Education curriculum (to be released) and the Japanese safety set AnswerCarefully (gated) (ev-techreport, ev-sft-dpo).

Function / tool calling

New in 4.1 and described by the publisher as the first LLM-jp series with tool calling (ev-blog). The training mix added a Nemotron agentic subset (21,048 samples) and a Japanese translation of it (11,174 samples), prioritising multi-step and parallel calls and excluding incorrect calls from the loss (ev-blog, ev-sft-dpo).

Evaluated by the publisher on BFCL (categories up to v3) and tau2-bench at medium reasoning effort; single-turn BFCL is equal to or slightly below Qwen3, tau2-bench is above Qwen3 and far below Qwen3.5, and multi-turn long-context is the weakest category (ev-blog, ev-card-41). The cookbook ships function-calling examples for Transformers and vLLM (ev-cookbook). The tool calls are parsed by publisher plugins; test the parser on your serving stack before use.

How this scores

The ownership factor this domain covers, drawn from the one entry record.

3

ReliabilityIs it reliable and good enough for the job?

Weak

Safety scores 2: one publisher-run Japanese benchmark, no independent evaluation or guard model, and the publisher says the models are not for direct production use; operational is 3 (ev-card-41, ev-blog, ev-cookbook).

How this scores (AOI sub-dimensions)
Operational3/5how practical it is to run, serve and maintain in productionRuns on Hugging Face Transformers, vLLM and llama.cpp with a maintained Apache-2.0 cookbook, official GGUF builds in BF16 and Q4_K_M, a 64K context and a Q4_K_M 33B validated on a local GPU.
Safety2/5whether misuse risks are evaluated and guardrails are providedThe 4.1 models are safety post-trained (the card says SFT and DPO "for instruction following and safety", and the report describes an upsampled Japanese safety set and safety-judged preference pairs), and the residual risk is disclosed.
What this means for adoptionYou own the weights, the data lineage and the right to run and change them: Apache-2.0, ungated, with public corpora, SFT and DPO sets and pre-training scripts, so this is one of the most transparent models reviewed. You do not own a deployable safety case: the publisher says the models are for research and development and not for direct production use, offers one self-run Japanese safety benchmark, no guard model and no independent test. Ownership moves to substantial if an independent evaluation lifts safety to 3 or more, and you can reach that yourself by running your own safety and jailbreak tests behind your own guard.

Sources

The same evidence records as the entry sheet. Read means the text was verified; unverified means it is known to exist but not yet read.

Model cardread2026-11-19
The llm-jp-4.1-33b-thinking model card declares Apache-2.0 in the front matter and 11.7T tokens; the 8B and 32B-A3B cards are identical on the points used.
Model cardread2026-11-19
The LLM-jp-4 (4.0) base, 33b-thinking and 8b-instruct cards say "The models released here are in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations".
Licenceread2026-11-19
Apache-2.0 is declared in the card front matter and License section of all three 4.1 cards and in section 2.4 of the technical report.
Technical_reportread2026-11-19
The LLM-jp-4 Technical Report (arXiv 2610.19331 v1, 2026-10-14) covers 8B dense and 32B-A3B MoE models trained from scratch on about 12T tokens.
Documentationread2026-11-19
LLM-jp Corpus v4.1 README: English FineWeb-Rescored and MegaMath-Web-Pro-Max-OSS (ODC-BY plus Common Crawl terms) and The Stack v2 code (per-repository licences), hosted on a public GitLab project; copyright stays with original authors; document-removal contact given.
Documentationread2026-11-19
LLM-jp Corpus v4 README lists Japanese, English, Korean, Chinese and code subcorpora with licences (mostly CC BY, ODC-BY, CC BY-SA).
Documentationread2026-11-19
LLM-jp Corpus Midtraining v2 README: 13 English and 2 Japanese sources with licences, including the nemotron_pretraining_code_v2 corpus (491B tokens) under the NVIDIA Data Access Agreement for Model Training and llm-jp-IPT_v0.3.2 under "Various licenses".
Documentationread2026-11-19
The llm-jp-4.1 SFT and DPO dataset cards (ungated) list per-subset licences, and responses were generated with gpt-oss-120b.
Documentationread2026-11-19
github.com/llm-jp/scripts (Apache-2.0) holds LLM-jp-4 pre-training and mid-training configurations (v4-8b, v4-32b which matches the 33B dimensions, v4-32b-a3.8b), installers, corpus-building and checkpoint-conversion scripts.
Documentationread2026-11-19
The Apache-2.0 llm-jp-4-cookbook documents Transformers 5.0.0 or later, vLLM 0.30.0 or later with llm-jp-vllm 0.1.1 or later, and llama.cpp v0.6.0 or later.
Model cardread2026-11-19
The official llm-jp-4.1-33b-thinking-gguf card (modified 2026-10-20) says running with llama.cpp "currently requires a fork of llama.cpp", which contradicts the cookbook; the repositories hold BF16 (66.4 GB) and Q4_K_M (20.4 GB) GGUF files and an imatrix.
Documentationread2026-11-19
The Hugging Face API lists the three 4.1 repositories as ungated, with safetensors BF16 shards carrying LFS sha256, no pickle files, no LICENSE file and a completed hub scan with no flagged files.
Documentationread2026-11-19
The NII release page lists the 4.1 models, corpora and fine-tuning scripts and says "Users of this model are requested to use it appropriately and at their own responsibility, in accordance with" Japan's December 2025 AI R&D and utilisation guideline.
Vendor announcementread2026-11-19
Publisher blog (2026-09-28, Japanese): 4.1 improves post-training for STEM and adds tool calling, the first LLM-jp series with it.
Third-party analysisread2026-11-19
PC Watch (Impress, 2026-09-29) reports that NII released the three LLM-jp-4.1 models on 2026-09-28 and that Apache License 2.0 applies to all.
Documentationunverified2026-11-19
The AnswerCarefully dataset (safety training data and benchmark) is gated (manual approval, licence:other); its LICENSE and README returned HTTP 401, so its terms are documented but unverified.