Ax Wenjie Jacky Mo, Qin Liu, Xiaofei Wen, Dongwon Jung, Hadi Askari, Wenxuan Zhou, Zhe Zhao, Muhao Chen 29d ago

RedCoder: Automated Multi-Turn Red Teaming for Code LLMs

RedCoder: Automated multi-turn red teaming system for identifying vulnerabilities in code generation LLMs without extensive human effort.

Ax Zhaoyi Joey Hou, Bowei Alvin Zhang, Yining Lu, Bhiman Kumar Baghel, Anneliese Brei, Ximing Lu, Meng Jiang, Faeze Brahman, Snigdha Chaturvedi, Haw-Shiuan Chang, Daniel Khashabi, Xiang Lorraine Li 29d ago

CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity

CreativityPrism: Cross-domain evaluation framework for measuring LLM creativity across diverse scenarios without heavy reliance on human evaluation.

Ax Jingsong Liu, Han Li, Zhengyang Xu, Franz-Leonard Klaus, Fabian St\"ogbauer, Shihui Zu, Weiwei Zhou, Atsuko Kasajima, Felix Schicktanz, Alexander Muckenhuber, Julius Shakhtour, Jiale Yu, Tiannan Zheng, Xun Ma, Maggie Wang, Christian Grashei, Bao Li, Guiyang Jiang, Hongming Xu, Shaohua Kevin Zhou, Nassir Navab, Peter J. Sch\"uffler 29d ago

Towards Cellular-Scale Interpretability in Pathology Foundation Models for Biomarker Assessment

Hireca pathology foundation model for interpretable biomarker assessment from histology images using pretrained vision-language architecture.

Ax Michael Ofengenden, Yunze Man, Ziqi Pang, Liang-Yan Gui, Yu-Xiong Wang 29d ago

PPTArena: A Benchmark for PowerPoint Editing

Benchmark with 1,300+ human-curated PowerPoint edits for evaluating agents' ability to modify slides from natural language instructions.

Ax Karolina Korgul, Yushi Yang, Arkadiusz Drohomirecki, Piotr B{\l}aszczyk, Will Howard, Lukas Aichberger, Chris Russell, Philip H. S. Torr, Adam Mahdi, Adel Bibi 29d ago

It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents

Benchmark for evaluating LLM-powered web agents' robustness to prompt injection attacks hidden in interface elements that redirect task completion.

Ax Lukas Sch\"afer, Pallavi Choudhury, Abdelhak Lemkhenter, Chris Lovett, Somjit Nath, Luis Fran\c{c}a, Matheus Ribeiro Furtado de Mendon\c{c}a, Alex Lamb, Riashat Islam, Siddhartha Sen, John Langford, Katja Hofmann, Sergio Valcarcel Macua 29d ago

When Does Predictive Inverse Dynamics Outperform Behavior Cloning?

Theoretical analysis comparing predictive inverse dynamics models to behavior cloning for imitation learning with limited expert demonstrations.

Ax Eden Saig, Tamar Garbuz, Ariel D. Procaccia, Inbal Talgam-Cohen, Jamie Tucker-Foltz 29d ago

Adaptive Contracts for Cost-Effective AI Delegation

Adaptive contracts framework for cost-effective AI delegation in text generation tasks with selective evaluation methods.

Ax Letian Fu, Justin Yu, Karim El-Refai, Ethan Kou, Haoru Xue, Huang Huang, Wenli Xiao, Guanzhi Wang, Dantong Niu, Fei-Fei Li, Guanya Shi, Jiajun Wu, Shankar Sastry, Yuke Zhu, Ken Goldberg, Linxi "Jim" Fan 29d ago

CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation

CaP-X framework for benchmarking Code-as-Policy agents in robot manipulation tasks using executable code and vision-language-action methods.

Ax Yujin Potter, Nicholas Crispino, Vincent Siu, Chenguang Wang, Dawn Song 29d ago

Peer-Preservation in Frontier Models

Study of misaligned behaviors in frontier AI models including peer-preservation behavior across eight models.

Ax Wanghan Xu, Shuo Li, Tianlin Ye, Qinglong Cao, Yixin Chen, Hengjian Gao, Yiheng Wang, Qi Li, Kun Li, Sheng Xu, Shengdu Chai, Fangchen Yu, Xiangyu Zhao, Zhangrui Zhao, Weijie Ma, Zijie Guo, Koutian Wu, Haoyu Zhou, Haoxiang Yin, Lixue Cheng, Chaofan Hu, Haoxuan Li, Lu Mi, Xuxuan Xie, Yifan Zhou, Ruizhe Chen, Zhiwang Zhou, Xingjian Guo, Yuhao Zhou, Xuming He, Shengyuan Xu, Xinyu Gu, Jiamin Wu, Mianxin Liu, Chunfeng Song, Fenghua Ling, Dongzhan Zhou, Shixiang Tang, Yuqiang Li, Mao Su, Peng Ye, Siqi Sun, Bin Wang, Xue Yang, Zhenfei Yin, Tianfan Fu, Guangtao Zhai, Wanli Ouyang, Bo Zhang, Lei Bai, Wenlong Zhang 29d ago

ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research

Benchmark for evaluating autonomous AI agents on 40 scientific research tasks across 10 domains grounded in published papers.