Qwen2 5 vl new vit

[yixqiao]

Add the new ViT class in vLLM to Qwen 2.5 VL, removing the huggingface pretrained dependency.

Includes changes to MLP, window-based partial attention, RMSNorm, when compared to 2 VL. Enables parallelized operations where appropriate.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[ywang96]

Thanks for this great PR! I left a few comments and please take a look at them!

In addition, it would be great if you can show some results on

1. Correctness verification on the ViT: We don't need to add a unit test on it but we should at least check if the embeddings generated from the same image match with the transformers implementation, in both TP=1 and TP>1 cases.
2. Speed verification: our implementation should be at least not slower than transformers implementation.

[ywang96]

@yixqiao Thank you for the work! I will take it over from here.