Jinpeng Lu | World Models, VLM/VLA, Medical AI

👨‍🎓 About Me

Hi, I am Jinpeng Lu, a 2025-entry master's student in Information and Communication Engineering at the University of Science and Technology of China. I work with Prof. Zhiwei Xiong. Previously, I received my B.S. in Mathematics from Harbin Institute of Technology, Shenzhen.

My current research focuses on world-state consistency in video generation models, especially under multi-shot generation, viewpoint changes, and camera movement. I am also actively exploring VLMs and AI agents, with interest in how multimodal systems represent scenes, preserve state, and support reliable reasoning.

My work combines benchmark design, representation analysis, and efficient model engineering. In medical vision, this has meant building lightweight 3D segmentation systems and testing how visual foundation models transfer across modalities. In world-model, VLM, and agent-related research, I focus on diagnostics that make long-horizon state, spatial grounding, and behavior traces measurable rather than anecdotal.

📢📢 I am currently exploring world-state consistency in video generation models, including multi-shot scenarios and camera motion, while actively expanding into VLMs and AI agents.

Open benchmark WRBench D1-D6 diagnostics · project page · HF artifacts · leaderboard
First-author papers ICLR 2026 · MICCAI 2024 · world-model diagnostics
Engineering base PyTorch · CUDA · evaluation pipelines · 3D perception

🔥 News

2026.06Current World Models Lack a Persistent State Core released with WRBench, including the project page, GitHub repo, HF artifacts, and leaderboard for D1-D6 world-state diagnostics.
2026.06Contributor outputs in embodied intelligence and VLA diagnosis include Pelican-Unify 1.0 and VLA-Trace.
2026.01Johnson-Lindenstrauss Lemma Guided Network for Efficient 3D Medical Segmentation accepted to ICLR 2026.
2024.03H2ASeg accepted to MICCAI 2024 as an early-accept paper.

📘 Selected Publications

Full list: Google Scholar h-index 4 · i10-index 3

🌍 World Model

arXiv 2026

WRBench source figure showing viewpoint intervention and returned-state evidence

First Author 2026.06

Current World Models Lack a Persistent State Core

J. Lu, D. Zhu, H. Shi, L. Cai, G. Tang, Y. Chen, J. Cao, D. Tang, Y. Zhang, Y. Dai, X. Ju.

Task: Evaluate whether video generation world models can preserve dynamic scene state across viewpoint changes, multi-shot observation, and camera movement.
Method: Build WRBench as a D1-D6 diagnostic protocol with camera-observability interventions, re-observation cases, public artifacts, and leaderboard-style evaluation.
Result: Shows that current world models often fail to maintain a persistent state core, making returned-state consistency measurable instead of anecdotal.

Paper Project Code Artifacts Leaderboard Source Figure

🧠 VLM & VLA Representation Learning

arXiv 2026

Pelican-Unify source figure showing embodied intelligence model components

Contributor 2026.06

Pelican-Unify 1.0: A Unified Embodied Intelligence Model

Task: Support a unified embodied-intelligence model that connects perception, reasoning, future prediction, and action generation.
Method: Contribute to the model loop and evaluation framing around scene understanding, task reasoning, future imagination, and action prediction.
Result: Helps organize embodied AI capabilities into a single pipeline for studying how multimodal representations drive downstream behavior.

Paper Source Figure

arXiv 2026

VLA-Trace source figure showing representation and behavior tracing framework

Contributor 2026.06

VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing

Task: Diagnose why vision-language-action models succeed or fail during embodied rollouts.
Method: Trace representation dynamics and behavior signals across model layers, decisions, and rollout outcomes.
Result: Provides a more inspectable path from internal VLA states to observed behavior, supporting failure analysis beyond final task scores.

Paper Source Figure

🏥 Medical Vision Foundation Models

ICLR 2026

First Author CCF A 2026.01

Johnson-Lindenstrauss Lemma Guided Network for Efficient 3D Medical Segmentation

J. Lu, L. Cai, Y. Chen, G. Tang, S. Jiang, H. Shi, Z. Xiong.

Task: Perform efficient multimodal 3D medical segmentation for volumetric PET/CT perception.
Method: Combine JL-guided convolution, paired window attention, and spatially decoupled knowledge transfer in VeloxSeg.
Result: Delivers a 1.66M-parameter segmentation model that keeps the architecture lightweight for 3D medical-image analysis.

Paper Code Source Figure

Tech Report 2025

Co-first Author 2025.10

Does DINOv3 Set a New Medical Vision Standard?

C. Liu*, Y. Chen*, H. Shi*, J. Lu*, B. Jian*, J. Pan*, L. Cai*, et al.

Task: Test whether DINOv3-style self-supervised visual features transfer reliably to medical vision tasks.
Method: Benchmark 2D and 3D classification, segmentation, and registration across multiple medical modalities.
Result: Identifies where transfer is useful, where task limits remain, and which modality gaps matter for medical foundation-model use.

Paper HF Paper Source Figure

MICCAI 2024

H2ASeg source figure showing PET/CT hierarchical adaptive interaction architecture

First Author CCF B 2024.03

H2ASeg: Hierarchical Adaptive Interaction and Weighting Network for Tumor Segmentation in PET/CT Images

J. Lu, J. Chen, L. Cai, S. Jiang, Y. Zhang.

Task: Segment tumor regions in paired PET/CT images under heterogeneous lesion appearance and modality imbalance.
Method: Use hierarchical adaptive interaction and weighting to fuse PET metabolic cues with CT anatomical structure.
Result: Improves robust lesion-region segmentation and establishes the medical-segmentation line that later led to VeloxSeg.

Paper Code Source Figure

Additional Medical AI Publications

AttriMIL: Revisiting Attention-based Multiple Instance Learning for Whole-slide Pathological Image Classification Medical Image Analysis 2025 · Contributor. Contributes to attention-based multiple-instance learning for whole-slide pathology image classification.
IPGPhormer: Interpretable Pathology Graph-Transformer for Survival Analysis arXiv 2025 · Contributor. Contributes to interpretable graph-transformer modeling for pathology-based survival analysis.
A Localization-to-Segmentation Framework for Automatic Tumor Segmentation in Whole-Body PET/CT Images arXiv 2023 · Contributor. Contributes to a localization-to-segmentation cascade for automatic whole-body PET/CT tumor segmentation.

→ Full List of Publications

🔬 Research

Current interests organized around world models, representation learning, and medical vision foundation models.

🌍 World Model

Dynamic world-state diagnostics under viewpoint changes.

I study whether video generation systems maintain coherent dynamic world states under viewpoint changes. In WRBench, camera motion is treated as an observability intervention, and the D1-D6 diagnostic chain separates camera execution, visual integrity, visible spatial/state consistency, re-observation support, and returned spatial/state consistency.

Returned-state evidence is a key diagnostic stage rather than the whole benchmark identity. This line focuses on benchmark design, human-calibrated evaluation, public artifact release, and failure analysis across model families, camera-control interfaces, and long-horizon visual consistency.

🧠 VLM & VLA Representation Learning

Video-language representations, action routing, and behavior tracing.

I am interested in how multimodal representations become embodied behavior. Recent contributor projects such as Pelican-Unify and VLA-Trace help connect scene understanding, reasoning, future imagination, action prediction, and rollout-level behavior.

The goal is to make VLA and embodied foundation models more inspectable: where task evidence appears, how it propagates, and when it fails to control action.

🏥 Medical Vision Foundation Models

Efficient 3D perception, multimodal segmentation, and transfer limits.

My earlier work develops efficient and robust segmentation methods for PET/CT and 3D medical images. In VeloxSeg, I combine JL-guided convolution, paired window attention, and spatially decoupled knowledge transfer for lightweight multimodal 3D segmentation.

I also study transfer limits of visual foundation models in medicine, including DINOv3-style self-supervised features across 2D/3D classification, segmentation, and registration tasks.

🎓 Education

2025.09 - 2028.06

University of Science and Technology of China

M.S. in Information and Communication Engineering. Advisor: Prof. Zhiwei Xiong.

2021.09 - 2025.06

Harbin Institute of Technology, Shenzhen

B.S. in Mathematics. GPA: 3.728 / 4, ranking 17 / 172.

🛠️ Methods / Engineering Profile

Evaluation Design

I build diagnostic benchmarks and protocols that separate task setup, observable evidence, judgeability, and model failure modes, especially for world models and embodied multimodal systems.

Model Engineering

I design efficient 3D perception modules, multimodal fusion blocks, and representation analyses in PyTorch/CUDA, with ablations tied to the specific behavior being measured.

Reproducible Systems

I maintain experiment pipelines, artifact releases, leaderboards, and data-processing workflows with Linux, Git, Docker, Shell/Bash, LaTeX, and experiment tracking tools.

👣 Site Visitors

Approximate visitor map and page-view counter.

Approximate Location

Approximate location: loading...

Location is estimated from public IP geolocation and may be imprecise; this page does not display your raw IP address.