when exchange partition, info schema report "failed to load schema diff" and do full load on "Found a row that does not match the partition" error

[D3Hunter]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

for the integration test db_partition.test, the exchange partition will fail with Found a row that does not match the partition, it's expected, but info schema report "failed to load schema diff" and do full load, this part is not.

CREATE TABLE t1 (
	d date NOT NULL ,
	name varchar(10)  NOT NULL,
	UNIQUE KEY (d,name)
);
CREATE TABLE t1p (
	d date NOT NULL ,
	name varchar(10)  NOT NULL,
	UNIQUE KEY (d,name)
)
PARTITION BY RANGE COLUMNS(d)
(PARTITION p202307 VALUES LESS THAN ('2023-08-01'),
 PARTITION p202308 VALUES LESS THAN ('2023-09-01'),
 PARTITION p202309 VALUES LESS THAN ('2023-10-01'),
 PARTITION p202310 VALUES LESS THAN ('2023-11-01'),
 PARTITION p202311 VALUES LESS THAN ('2023-12-01'),
 PARTITION p202312 VALUES LESS THAN ('2024-01-01'),
 PARTITION pfuture VALUES LESS THAN (MAXVALUE));
insert into t1 values ("2023-08-06","0000");
-- error 1737
alter table t1p exchange partition p202307 with table t1 with validation;

[2024/10/14 18:06:08.766 +08:00] [ERROR] [domain.go:356] ["failed to load schema diff"] [error="[schema:1146]Table '(Schema ID 2887).(Table ID 3156)' doesn't exist"]
[2024/10/14 18:06:08.786 +08:00] [INFO] [domain.go:392] ["full load InfoSchema success"] [isV2=true] [currentSchemaVersion=5616] [neededSchemaVersion=5617] ["elapsed time"=20.049449ms]

2. What did you expect to see? (Required)

error, but no full load

3. What did you see instead (Required)

error and full load

4. What is your TiDB version? (Required)

master