Document Type
Poster
Publication Date
Fall 2025
Abstract
This study benchmarks four leading LLMs (GPT-5.2, Claude 4.5, Gemini 3, and Grok-4) on cross-cultural advertising transcreation, using Turkish as a particularly revealing test case. Each model generated Turkish billboard taglines for three American brands (Hinge, Manscaped, and Oatly) under real-world constraints, and all outputs were scored using the Levent Index, a 100-point framework I created by synthesizing transcreation best practices with the Turkish linguistic instincts I developed growing up in Istanbul. The Index measures Linguistic Soul (40%), Cultural Resonance (40%), and Structural Flow (20%), and the complete rubric is available upon request. Gemini 3 dominated performance, yet no model exceeded 75 points. What was surprising is that all models handled the mechanical aspects of Turkish (pronoun economy, compression, syllabic rhythm) reasonably well, but consistently fell short on the creative and cultural dimensions, the ability to not just sound local, but feel local. Grok-4 catastrophically scored 1 point on Oatly's sarcastic tone, while GPT-5.2 underperformed despite its market dominance. These findings point to significant gaps in multilingual creative capabilities that likely extend well beyond Turkish.
Recommended Citation
Kocaman, Murathan, "Lost in Transcreation: LLM Performance in Multicultural Creative Work" (2025). IPHS 200: Programming Humanity. Paper 92.
https://digital.kenyon.edu/dh_iphs_prog/92
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.