- Science & Technology
- Understanding the Effects of Technology on Economics and Governance
As generative foundation models (LLMs, text-to-image, and vision-language models) are deployed in increasingly high-stakes settings, knowing how much to trust them matters more than ever. Yet existing evaluations are fragmented, quickly outdated, and often limited to a single modality.
TrustGen provides a unified taxonomy with 25+ fine-grained trust dimensions (truthfulness, safety, fairness, robustness, privacy, machine ethics) evaluated consistently across modalities. TrustGen uses the taxonomy for dynamic benchmarking with a modular pipeline (Metadata Curator → Test Case Builder → Contextual Variator) that automatically generates fresh test cases to stay ahead of model evolution and prevent memorization.
The publication evaluates 39 models tested (8 text-to-image, 21 LLMs, 10 vision-language models), surfacing key insights:
- Open-source models now rival proprietary ones on trustworthiness,
- hallucination, fairness, and privacy remain the biggest gaps,
- and trustworthiness has a "ripple effect" on helpfulness and ethics.
Great collaboration led by Yue Huang (University of Notre Dame) across academia and industry with SISLers Anka Reuel and Max Lamparth, Ph.D.
Paper: https://lnkd.in/g-Nnc-g7
Cite the Paper [APA]:
Huang, Y., Gao, C., Wu, S., Wang, H., Wang, X., Ye, J., Zhou, Y., Wang, Y., Shi, J., Zhang, Q., Bao, H., Liu, Z., Li, Y., Guan, T., Wang, P., Zhuang, H., Chen, D., Guo, K., Zou, A., Hooi, B., Xiong, C., Stengel-Eskin, E., Zhang, H., Yin, H., Zhang, H., Yao, H., Zhang, J., Yoon, J., Shu, K., Krishna, R., Swayamditta, S., Shi, W., Li, X., Hao, Y., Jia, Z., Li, Z., Chen, X., Tu, Z., Hu, X., Zhou, T., Zhao, J., Sun, L., Huang, F., Cohen-Sasson, O., Sattigeri, P., Reuel, A., Lamparth, M., Zhao, Y., Dziri, N., Su, Y., Sun, H., Ji, H., Xiao, C., Bansal, M., Chawla, N. V., Pei, J., Gao, J., Backes, M., Yu, P. S., Gong, N. Z., Chen, P.-Y., Li, B., Song, D., & Zhang, X. (2026). TrustGen: A platform of dynamic benchmarking on the trustworthiness of generative foundation models. In Proceedings of the Fourteenth International Conference on Learning Representations (ICLR 2026).
