CORE-15 feat: Support Qwen3.5 text generation models#12771
Merged
comfyanonymous merged 11 commits intoComfy-Org:masterfrom Mar 26, 2026
Merged
CORE-15 feat: Support Qwen3.5 text generation models#12771comfyanonymous merged 11 commits intoComfy-Org:masterfrom
comfyanonymous merged 11 commits intoComfy-Org:masterfrom
Conversation
kijai
requested review from
Kosinkadink,
comfyanonymous and
guill
as code owners
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds full QWEN35 support (new comfy/text_encoders/qwen35.py and tokenizer config) and five TEModel.QWEN35_* enum variants. Extends TE detection and loading to recognize and route QWEN35 weights, updates single- and multi-SD TE routing to instantiate qwen35 TEs and tokenizers, and normalizes state_dict prefixes. Propagates a new presence_penalty parameter through CLIP/SD clip generate methods, Llama2 generate/sample_token paths, and node-level text-generation inputs. Introduces Llama2 helpers (get_past_len, compute_freqs_cis, init_kv_cache). Updates llama_detect to check multiple weight keys. 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@comfy_extras/nodes_textgen.py`:
- Line 37: The node schema exposes an input named "thinking" but
TextGenerateLTX2Prompt.execute does not accept or forward it; update the execute
method signature to accept a thinking parameter (with same optional/default
semantics) and ensure the thinking value is forwarded to wherever other generate
parameters are passed (e.g., the call that builds or calls the LTX2
prompt/generation function), referencing TextGenerateLTX2Prompt.execute and the
"thinking" input so the node interface remains consistent and the option is
effective.
In `@comfy/sd.py`:
- Around line 429-435: The generate wrapper in comfy/sd.py changed parameter
order (placing presence_penalty before seed) which breaks callers that pass seed
positionally; update the generate method signature in the class (def
generate(...)) so that seed appears in the original position (move seed before
presence_penalty and any subsequent params) or restore the original parameter
ordering, and ensure the internal call to self.cond_stage_model.generate
continues to forward seed (use seed=seed to be explicit) — adjust the signature
and any callers of comfy.sd.generate accordingly to preserve backward
compatibility.
In `@comfy/sd1_clip.py`:
- Around line 743-744: The generate method's signature changed positional
ordering by adding presence_penalty after top_p but before seed, breaking
callers that pass seed positionally; restore backward compatibility by keeping
seed as the last positional parameter (or accept *args/**kwargs) so existing
positional calls to generate(..., seed) still bind correctly—update the generate
function signature in comfy/sd1_clip.py (method generate) to place seed before
any newly added optional params or refactor to forward **kwargs to getattr(self,
self.clip).generate while preserving the original parameter order.
In `@comfy/text_encoders/qwen35.py`:
- Line 47: The stop_tokens default only contains 248044 and therefore omits the
tokenizer's configured EOS token 248046 (`<|im_end|>`), causing generation to
potentially ignore normal EOS; update the default_factory for stop_tokens in the
stop_tokens field (where it's currently set to [248044]) to include 248046 as
well so generation will stop on the configured EOS token; ensure any tests or
callers that rely on stop_tokens still behave correctly after adding the EOS id.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 782807b1-5316-416a-9541-24840891ed18
📒 Files selected for processing (8)
comfy/sd.pycomfy/sd1_clip.pycomfy/text_encoders/llama.pycomfy/text_encoders/qwen35.pycomfy/text_encoders/qwen35_tokenizer/merges.txtcomfy/text_encoders/qwen35_tokenizer/tokenizer_config.jsoncomfy/text_encoders/qwen35_tokenizer/vocab.jsoncomfy_extras/nodes_textgen.py
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@comfy/sd.py`:
- Around line 1253-1263: The detection code currently assumes the presence of
"model.language_model.layers.0.input_layernorm.weight" after checking
"model.language_model.layers.0.linear_attn.A_log", which can raise KeyError for
variant checkpoints; update the branch that checks
"model.language_model.layers.0.linear_attn.A_log" to first verify the existence
of "model.language_model.layers.0.input_layernorm.weight" (e.g., using
sd.get(...) or an explicit key in sd check) and only read weight.shape if
present, otherwise fall back to the default TEModel (TEModel.QWEN35_2B) or an
appropriate safe branch; ensure the change references the same keys and TEModel
constants (TEModel.QWEN35_08B, TEModel.QWEN35_4B, TEModel.QWEN35_9B,
TEModel.QWEN35_27B, TEModel.QWEN35_2B) so detection remains backward-compatible.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 9fc56ddd-cbf0-4236-8116-9acabb07e511
📒 Files selected for processing (4)
comfy/sd.pycomfy/sd1_clip.pycomfy/text_encoders/qwen35.pycomfy_extras/nodes_textgen.py
🚧 Files skipped from review as they are similar to previous changes (1)
- comfy/text_encoders/qwen35.py
|
This comment is a feature request loosely related to this PR, please disregard it if it's out of place. I wanted at some point (but I neglected) to make a feature request for a node similar to Or, even better, a more generally applicable prompt enhancer node with a system prompt that can be edited by the user. |
ComfyUI Error ReportError Details
Stack TraceSystem Information
Devices
|
Contributor
Author
This would be issue for https://github.com/city96/ComfyUI-GGUF , I don't think it supports Qwen3.5. |
|
@kijai Even with the same settings and model as you, I still get errors. What should I do? 
ComfyUI Error ReportError Details
Stack TraceSystem Information
Devices
|
Contributor
Author
This PR isn't merged yet, based on the error you haven't merged it manually either so it won't recognize the model. Either wait until it's merged or apply the PR yourself for testing. |
Thank you! |
|
Waiting to be merged into the main branch |
|
This actually working? 
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@comfy_extras/nodes_textgen.py`:
- Around line 27-29: Restore the original node category string for the
TextGenerate node to avoid unintentionally moving UI entries: change the
category value in the Schema returned by the TextGenerate node (node_id
"TextGenerate") back to the previous "textgen/" form instead of "textgen" so
that its child/inheriting node TextGenerateLTX2Prompt keeps the same menu
placement; ensure any other nodes relying on that category are unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 4d6e7be3-7c8c-410c-a689-8c0ce71af0b8
📒 Files selected for processing (1)
comfy_extras/nodes_textgen.py
|
Any info of this will be merged? checked locally on my comfy and it's working great |
|
In my experience so far, the TextGenerate node of ComfyUI takes extremely too much time to complete compared to connecting to a server running locally any LLM model (big or small). In fact, using connection I can get 60-150k/s in some models while no matter what model I choose for TextGenerate it takes more than 100s for 250 tokens! |
@Dev0Lab1 It should not be that slow. On my system I get 90t/s using llama.cpp (f16 gguf) and 40t/s (bf16) on ComfyUI. |
|
Some generous soul who can share where to find the 9B version of Qwen3.5? Or at least share the script to convert it for ComfyUI |
Contributor
Author
I have it ready, just didn't upload yet. |
alexisrolland
changed the title
I get the same error message running this in ComfyUI 0.8.28 (26040356464hyh4) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds support for Qwen 3.5 text generation models.
Model for testing (only uploaded 4b in case changes to the layer names are requested):
https://huggingface.co/Comfy-Org/Qwen3.5
Tested with 2b, 4b, 9b