TiDB doesn't deduplicate the argument of the IN function

[winoros]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

create table t(a int);

explain select * from t where a in (1,1,1,1,1,1,1);

2. What did you expect to see? (Required)

the duplicate 1 should be eliminated

3. What did you see instead (Required)

mysql> explain select * from t where a in (1,1,1,1,1,1,1);
+----------------------------+----------+--------------+---------------+----------------------------------------------------------+
| id                         | estRows  | task         | access object | operator info                                            |
+----------------------------+----------+--------------+---------------+----------------------------------------------------------+
| TableReader_16             | 10.00    | root         |               | MppVersion: 3, data:ExchangeSender_15                    |
| └─ExchangeSender_15        | 10.00    | mpp[tiflash] |               | ExchangeType: PassThrough                                |
|   └─Selection_14           | 10.00    | mpp[tiflash] |               | in(test.t.a, 1, 1, 1, 1, 1, 1, 1)                        |
|     └─TableFullScan_13     | 10000.00 | mpp[tiflash] | table:t       | pushed down filter:empty, keep order:false, stats:pseudo |
+----------------------------+----------+--------------+---------------+----------------------------------------------------------+
4 rows in set (0.01 sec)

4. What is your TiDB version? (Required)

current master

[winoros]

The actual execution of all execution(tidb, tiflash, tikv) is based on a hash map. So the execution perf is guaranteed.

the problem is that if the user pass too many items, the size of grpc message will be unexpected.

[winoros]

This issue arises because we just suffer from one user sending a very large gprc message caused by some errors on his application side. This causes tikv's grpc pool to use high memory.