Skip to content

nft collections es improvements#1530

Merged
bogdan-rosianu merged 9 commits intodevelopmentfrom
API-699-nft-collections-es-improvements
Sep 4, 2025
Merged

nft collections es improvements#1530
bogdan-rosianu merged 9 commits intodevelopmentfrom
API-699-nft-collections-es-improvements

Conversation

@cfaur09
Copy link
Copy Markdown
Contributor

@cfaur09 cfaur09 commented Aug 26, 2025

Reasoning

Proposed Changes

How to test

@cfaur09 cfaur09 self-assigned this Aug 26, 2025
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Aug 26, 2025

k6 load testing comparison.
Base Commit Hash: 70ffe7e
Target Commit Hash: f1eb2d0

Metric Base Target Diff
AvgMax9095AvgMax9095AvgMax9095
Transactions79.143032.1163.8880.18111.412714.1777.5785.63+40.77% 🔴-10.49% ✅+21.42% 🔴+6.79% 🔴
Nodes60.341347.3962.8165.6665.51939.9675.1777.80+8.58% 🔴-30.24% ✅+19.68% 🔴+18.48% 🔴
Mex58.80393.0962.6165.6564.07298.3175.4978.45+8.95% 🔴-24.11% ✅+20.57% 🔴+19.49% 🔴
Pool59.13607.4762.6765.9064.34376.1075.4978.61+8.81% 🔴-38.09% ✅+20.46% 🔴+19.29% 🔴
Tokens59.34733.6362.6765.8564.79661.4075.4878.55+9.17% 🔴-9.85% ✅+20.44% 🔴+19.28% 🔴
Blocks69.141470.6463.2669.6573.411210.7676.8082.20+6.16% 🔴-17.67% ✅+21.41% 🔴+18.01% 🔴
Accounts60.531505.5262.7665.7865.951251.4375.0477.92+8.96% 🔴-16.88% ✅+19.55% 🔴+18.47% 🔴
Test Run Duration60002.9060002.02

Legend: Avg - Average Response Time, Max - Maximum Response Time, 90 - 90th Percentile, 95 - 95th Percentile
All times are in milliseconds.

@bogdan-rosianu bogdan-rosianu merged commit 3909b76 into development Sep 4, 2025
9 of 12 checks passed
@bogdan-rosianu bogdan-rosianu deleted the API-699-nft-collections-es-improvements branch September 4, 2025 08:04
bogdan-rosianu added a commit that referenced this pull request Sep 5, 2025
* Enhance MexTokenService to calculate and store token volumes for base and quote tokens in getAllMexTokensRaw method

* extend aws s3 config

* update configs

* update s3 configs

* fix transfers count with relayed txs (#1496)

* add support for new NFT events

* add maximum process retries

* add unit tests

* ferch token price from highest liquidity pool

* emit also deleteCacheKeys

* undo process reries

* fix tests

* Proposer fix (#1498)

* proposer fix

* proposer fix

* proposer fix

* fix getBlock

* fixes after review

Co-authored-by: bogdan-rosianu <bogdan.rosianu@yahoo.com>

* add execution order sorting using miniBlocksDetails from Elasticsearch (#1504)

* add execution order sorting using miniBlocksDetails from Elasticsearch

* fixes after review

* fixes after review

* fix userUndelegatedList field (#1507)

* fix userUndelegatedList field

* use ?? instead of [] to handle undefined case

* collections count subType required false fix (#1509)

* tokens supply format from plugin (#1505)

* fix collection set fields (#1510)

* fix collection set fields

* fixes after review

* add configurable response compression with gzip deflate support + update package json (#1512)

* add configurable response compresion

* set true compression for testnet env

* refactor processNFtCollections method

* update package

* update version of node

* update unit test yaml

* getNfts parallel execution

* compression flag false by default

* improve getSingleNFt

* getNftsCollection increase performance

* undo last commit

* Integrate last sdk nestjs (#1516)

* update to latest sdk version

* fixes for latest sdk version

* add package-lock

* fix unit tests

* Packages security issues 2 (#1517)

* fix packages

* add package-lock

* add missing packages

* remove unneeded package

* fix mongo connection

* Api 672 add timestampMs field into transaction response (#1518)

* Add timestampMs field to Transaction interface and class

* update specs

* update accounts spec

* Add normalizeTimestampMs method for timestamp conversion (#1519)

* Enhance account filtering by adding 'withBalance' query option to account endpoints and update ElasticIndexerHelper to support balance filtering. This allows users to filter accounts based on their balance status. (#1521)

* add events filtered by order (#1523)

* Refactor content type validation in NftMediaService to handle media types correctly by stripping parameters from the content type string. (#1527)

* use events index instead of logs (#1514)

* use events index instead of logs

* configurable migrated indices

* added protection for empty fields

* fix log address

* add transaction get unit tests (#1525)

---------

Co-authored-by: cfaur09 <catalinfaurpaul@gmail.com>
Co-authored-by: Catalin Faur <52102171+cfaur09@users.noreply.github.com>

* fix ESDTTransfer duplicated events (#1526)

* use events index instead of logs

* configurable migrated indices

* added protection for empty fields

* fix log address

* add transaction get unit tests (#1525)

* fix duplicated events

---------

Co-authored-by: bogdan-rosianu <bogdan.rosianu@yahoo.com>

* update MetaESDT tokens (#1520)

* update MetaESDT tokens

* update imports

* nft collections es improvements (#1530)

* ES collections improvements

* getNfts improvements

* improve bulkProcessTransactions

* fixes

* fixes

* fixes

* refactor getSmartContractResultsRaw()

* improve nftCollection processing

* eslint fix

---------

Co-authored-by: cfaur09 <catalinfaurpaul@gmail.com>
Co-authored-by: Gabriel Matei <gabrielvmatei@gmail.com>
Co-authored-by: bogdan-rosianu <51945539+bogdan-rosianu@users.noreply.github.com>
Co-authored-by: bogdan-rosianu <bogdan.rosianu@yahoo.com>
Co-authored-by: Catalin Faur <52102171+cfaur09@users.noreply.github.com>
Co-authored-by: Gutica Stefan <123564494+GuticaStefan@users.noreply.github.com>
bogdan-rosianu added a commit that referenced this pull request Sep 23, 2025
* update andromeda config template enable epoch (#1502)

* Merge development into main (#1511)

* Enhance MexTokenService to calculate and store token volumes for base and quote tokens in getAllMexTokensRaw method

* extend aws s3 config

* update configs

* update s3 configs

* fix transfers count with relayed txs (#1496)

* add support for new NFT events

* add maximum process retries

* add unit tests

* ferch token price from highest liquidity pool

* emit also deleteCacheKeys

* undo process reries

* fix tests

* Proposer fix (#1498)

* proposer fix

* proposer fix

* proposer fix

* fix getBlock

* fixes after review

Co-authored-by: bogdan-rosianu <bogdan.rosianu@yahoo.com>

* add execution order sorting using miniBlocksDetails from Elasticsearch (#1504)

* add execution order sorting using miniBlocksDetails from Elasticsearch

* fixes after review

* fixes after review

* fix userUndelegatedList field (#1507)

* fix userUndelegatedList field

* use ?? instead of [] to handle undefined case

* collections count subType required false fix (#1509)

* tokens supply format from plugin (#1505)

* fix collection set fields (#1510)

* fix collection set fields

* fixes after review

* add configurable response compression with gzip deflate support + update package json (#1512)

* add configurable response compresion

* set true compression for testnet env

* refactor processNFtCollections method

* update package

* update version of node

* update unit test yaml

* getNfts parallel execution

* compression flag false by default

* improve getSingleNFt

* getNftsCollection increase performance

* undo last commit

* Integrate last sdk nestjs (#1516)

* update to latest sdk version

* fixes for latest sdk version

* add package-lock

* fix unit tests

* Packages security issues 2 (#1517)

* fix packages

* add package-lock

* add missing packages

* remove unneeded package

* fix mongo connection

* Api 672 add timestampMs field into transaction response (#1518)

* Add timestampMs field to Transaction interface and class

* update specs

* update accounts spec

* Add normalizeTimestampMs method for timestamp conversion (#1519)

* Enhance account filtering by adding 'withBalance' query option to account endpoints and update ElasticIndexerHelper to support balance filtering. This allows users to filter accounts based on their balance status. (#1521)

* add events filtered by order (#1523)

* Refactor content type validation in NftMediaService to handle media types correctly by stripping parameters from the content type string. (#1527)

* use events index instead of logs (#1514)

* use events index instead of logs

* configurable migrated indices

* added protection for empty fields

* fix log address

* add transaction get unit tests (#1525)

---------

Co-authored-by: cfaur09 <catalinfaurpaul@gmail.com>
Co-authored-by: Catalin Faur <52102171+cfaur09@users.noreply.github.com>

* fix ESDTTransfer duplicated events (#1526)

* use events index instead of logs

* configurable migrated indices

* added protection for empty fields

* fix log address

* add transaction get unit tests (#1525)

* fix duplicated events

---------

Co-authored-by: bogdan-rosianu <bogdan.rosianu@yahoo.com>

* update MetaESDT tokens (#1520)

* update MetaESDT tokens

* update imports

* nft collections es improvements (#1530)

* ES collections improvements

* getNfts improvements

* improve bulkProcessTransactions

* fixes

* fixes

* fixes

* refactor getSmartContractResultsRaw()

* improve nftCollection processing

* eslint fix

---------

Co-authored-by: cfaur09 <catalinfaurpaul@gmail.com>
Co-authored-by: Gabriel Matei <gabrielvmatei@gmail.com>
Co-authored-by: bogdan-rosianu <51945539+bogdan-rosianu@users.noreply.github.com>
Co-authored-by: bogdan-rosianu <bogdan.rosianu@yahoo.com>
Co-authored-by: Catalin Faur <52102171+cfaur09@users.noreply.github.com>
Co-authored-by: Gutica Stefan <123564494+GuticaStefan@users.noreply.github.com>

* merge main to development - fixes after merge

* fix lint

---------

Co-authored-by: Rebegea Dragos-Alexandru <42241923+dragos-rebegea@users.noreply.github.com>
Co-authored-by: cfaur09 <catalinfaurpaul@gmail.com>
Co-authored-by: Gabriel Matei <gabrielvmatei@gmail.com>
Co-authored-by: Catalin Faur <52102171+cfaur09@users.noreply.github.com>
Co-authored-by: Gutica Stefan <123564494+GuticaStefan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants