fix(llm) : Streaming output generation and update gradio to latest version#197
fix(llm) : Streaming output generation and update gradio to latest version#197Aryankb wants to merge 11 commits intoapache:mainfrom
Conversation
|
@imbajin |
@Aryankb Thank you for your attempt, we already implement a partially less elegant asynchronous streaming output by modifying the previous version. We can keep this implementation first and let the current PR only upgrade the gradio version. If possible, a simple list of the useful features of 4.x -> 5.x for us is better. |
|
@imbajin okay, so i just need to list the possible changes (useful features) or need to make changes in gradio? Also, |
refer #190 (Added async func calls to the entire path, but some places have TODO) After introducing the async workflow framework, we should make all current operator/step calls async. Currently, there are many synchronous places, so it is very troublesome to change them |
Close this PR due to the function already supported & we move to |
Fixes #173
Fixes:-
rag_block -> rag_answer, can be applied to all other output boxes of gradioModifications:-
used gr.update() in rag_block to generate streaming output.
How to Test:-
Run python3 -m hugegraph_llm.demo.rag_demo.app in CLI
click
Answer questionunderRAG and user functions.the output is streaming. (though it is not async. the final output is streaming after collecting all outputs)
Screencast from 03-07-2025 01:13:46 PM.webm