Skip to content

chore(llm): using nuitka to provide a binary/perf way for the service#242

Merged
imbajin merged 8 commits intoapache:mainfrom
weijinglin:nk
May 21, 2025
Merged

chore(llm): using nuitka to provide a binary/perf way for the service#242
imbajin merged 8 commits intoapache:mainfrom
weijinglin:nk

Conversation

@weijinglin
Copy link
Copy Markdown
Collaborator

@weijinglin weijinglin commented May 19, 2025

follow #199 to build a new docker image using Nuitka in Dockerfile.nk

  • Only simple tests have been conducted so far
  • more complete tests are still needed

close #200

@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label May 19, 2025
@github-actions github-actions Bot added the llm label May 19, 2025
@dosubot dosubot Bot added the dependencies Pull requests that update a dependency file label May 19, 2025
@imbajin imbajin requested a review from Copilot May 20, 2025 03:18
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new Docker image build process using Nuitka for a binary/embed release along with enhancements to batch text embedding methods across multiple embedding providers.

  • Removed threaded processing in favor of a batched embedding API call in build_semantic_index.py
  • Added new get_texts_embeddings methods with documentation in embedding provider implementations (qianfan, openai, ollama)
  • Updated dependency versions for Ollama in requirements.txt and pyproject.toml, and added a new Dockerfile.nk for the build process

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
hugegraph_llm/operators/index_op/build_semantic_index.py Removed ThreadPoolExecutor and switched to using batched embedding API
hugegraph_llm/models/embeddings/qianfan.py Added batched embedding method with new docstring
hugegraph_llm/models/embeddings/openai.py Added batched embedding method with comprehensive docstring
hugegraph_llm/models/embeddings/ollama.py Updated API usage for embeddings and added batched embedding method
hugegraph_llm/models/embeddings/base.py Added abstract method for batched embedding to maintain consistency
hugegraph-llm/requirements.txt, pyproject.toml Updated Ollama dependency version
docker/Dockerfile.nk Added new Dockerfile using Nuitka for binary/embed release
Comments suppressed due to low confidence (3)

hugegraph_llm/models/embeddings/qianfan.py:55

  • [nitpick] Consider revising the docstring phrasing for clarity, e.g., changing 'Usage refer:' to 'See:' or 'Usage:' to align with standard documentation practices.
        """ Usage refer: https://cloud.baidu.com/doc/WENXINWORKSHOP/s/hlmokk9qn"""

hugegraph_llm/operators/index_op/build_semantic_index.py:40

  • The change from using ThreadPoolExecutor to a direct batched API call assumes that get_texts_embeddings scales well with larger lists. Please verify that this approach performs adequately for high-volume inputs.
embeddings = self.embedding.get_texts_embeddings(vids)

hugegraph_llm/models/embeddings/ollama.py:43

  • The update from self.client.embeddings to self.client.embed reflects the new API version. Please ensure that the parameters and expected return structure are fully aligned with the updated Ollama documentation.
return list(self.client.embed(model=self.model, input=text)["embeddings"][0])

Comment thread docker/Dockerfile.nk
@weijinglin weijinglin changed the title chore(production): using nk to provide a binary/embed way to release the service build(llm): using nk to provide a binary/embed way to release the service May 21, 2025
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels May 21, 2025
@imbajin imbajin changed the title build(llm): using nk to provide a binary/embed way to release the service chore(llm): using nuitka to provide a binary/perf way for the service May 21, 2025
@github-actions github-actions Bot removed the llm label May 21, 2025
@weijinglin
Copy link
Copy Markdown
Collaborator Author

image
The product of image constructed by Dockerfile.nk

Copy link
Copy Markdown
Member

@imbajin imbajin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to attach a tree structure for it

merge it & add docs for it later

@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label May 21, 2025
@imbajin imbajin merged commit 8887a3b into apache:main May 21, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

provide a binary/embed way to release the service

3 participants