A Semiotic-Based Usability Framework for Evaluating AI-Generated Visual Communication in Design Education: A UI/UX Testing Approach

Article Sidebar

PDF
Published: Jun 23, 2026
DOI: https://doi.org/10.34128/jsi.v12i1.1878
Keywords:
usability testing, semiotic UX, AI-generated design, komunikasi visual, mixed methods

Main Article Content

Nashiruddin Al-Fath
Binus University
Yufiarti Yufiarti
Psikologi, Fakultas Psikologi, Universitas Negeri Jakarta

Abstract

Adopsi platform AI generatif dalam praktik Desain Komunikasi Visual memunculkan pertanyaan mendasar yang belum terjawab secara empiris: apakah visual yang dihasilkan AI benar-benar mengomunikasikan makna yang dimaksud, ataukah sekadar mencapai daya tarik estetis? Penelitian ini mengembangkan dan memvalidasi Semiotic UX Evaluation Framework (SUEF), sebuah protokol evaluasi tujuh dimensi yang mengoperasionalkan teori semiotika sebagai metrik terukur dalam usability testing berbasis perangkat lunak. Menggunakan mixed methods sequential explanatory, 32 mahasiswa DKV dievaluasi melalui System Usability Scale, AttrakDiff, eye tracking, think-aloud protocol, dan pipeline NLP berbasis IndoBERT dan LDA. Hasil menunjukkan visual AI-generated memperoleh skor usabilitas kategori acceptable (SUS = 71.4) dibandingkan desain manual yang mencapai good (SUS = 81.2), dengan perbedaan signifikan (p < .001, Cohen's d = 0.85). Semiotic scoring mengungkapkan bahwa kesenjangan keduanya tidak bersifat estetis — Aesthetic Coherence hanya berbeda 0.2 poin — melainkan semiotik, terkonsentrasi pada Cultural Relevance (selisih 3.6) dan Intentionality Alignment (selisih 3.4). Temuan ini mendemonstrasikan bahwa AI generatif telah mencapai paritas estetis namun belum mencapai paritas komunikatif. SUEF tervalidasi sebagai protokol evaluasi yang dapat direplikasi untuk menilai kualitas komunikatif konten visual berbasis AI secara sistematis.

Article Details

How to Cite
[1]
N. Al-Fath and Y. Yufiarti, “A Semiotic-Based Usability Framework for Evaluating AI-Generated Visual Communication in Design Education: A UI/UX Testing Approach”, JSI, vol. 12, no. 1, pp. 70–79, Jun. 2026.
Issue
Vol. 12 No. 1 (2026): Jurnal Sains dan Informatika
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Author Biographies

Nashiruddin Al-Fath, Binus University

Nashiruddin Alfath, S.Sn., M.Sn., Lahir di Jakarta. Meraih gelar Sarjana Seni di Program Studi Animasi pada Fakultas Disain Komunikasi Visual tahun 2016. Meraih Magister Seni (MSn), Penciptaan Seni, di Institut Kesenian Jakarta tahun 2018. Saat ini penulis berprofesi sebagai dosen LLDIKTI Wilayah III dipekerjakan pada Universitas Bina Nusantara, Fakultas Desain Komunikasi Visual.

Yufiarti Yufiarti, Psikologi, Fakultas Psikologi, Universitas Negeri Jakarta

Prof Dr. Yufiarti, M.Psi. Lahir di Jakarta. Meraih sarjana Psikologi Pendidikan di IKIP Jakarta (Sekarang Universitas Negeri Jakarta) tahun 1986. Magister Psikologi di Universitas Indonesia Fakultas Psikologi tahun 1991. Doktor Pendidikan di IKIP Jakarta (UNJ) tahun 1996.  Saat ini berprofesi sebagai dosen di Fakultas Psikologi, Universitas Negeri Jakarta.

References

[1] F. De Saussure, Course in General Linguistics. New York: McGraw-Hill Book Company, 1996.

[2] R. M. Gazoni, “A Semiotic Analysis of Programming Languages,” pp. 91–101, 2018, doi: 10.4236/jcc.2018.63007.

[3] Y. Qian, Q. Bao, S. Zhang, and X. Peng, “A cultural memory semiotics and function behavior structure model for digital inheritance and innovation in AI generated Huizhou woodcarving images,” Sci. Rep., 2026, doi: https://doi.org/10.1038/s41598-026-35360-5.

[4] M. Tomalin, “Multimodal social semiotics and the challenge of arti fi cial intelligence,” Multimodality Soc., vol. 5, no. 2, pp. 223–244, 2025, doi: 10.1177/26349795251327939.

[5] M. G. Dondero, Semiotics of arti fi cial intelligence : enunciative praxis in image analysis and generation. 2025.

[6] M. Cang, Semiotics-AHP-Generative AI Pipeline for Intelligent Generation and Design of Shimao Stone-Carving Motifs, vol. 1, no. 1. Association for Computing Machinery, 2023.

[7] R. Berthes, Mythologies. New York, 1972.

[8] M. Thellefsen, B. Sørensen, and A. N. Dewi, “Generative AI and the semiosic reconfiguration of knowledge organization – a preliminary exploration,” J. Doc., vol. 81, pp. 1181–1199, Aug. 2025, doi: 10.1108/JD-04-2025-0094.

[9] L. Macnaught, “Customising chatbots for writing development: Anticipating semiotic mediation with the theoretical architecture of systemic functional linguistics Lucy,” J. English Acad. Purp., vol. 80, no. October 2025, p. 101646, 2026, doi: 10.1016/j.jeap.2026.101646.

[10] R. V Kozinets, “Netnography: Understanding Networked Communication Society,” in Handbook of Social Media Research Method, Toronto: York University, 2022, pp. 1–26.

[11] J. W. Creswell, Research Design, Qualitative, Quantitative, and Mixed Method Approaches,. California: SAGE Publications Inc., 2014.

[12] V. Braun and V. Clarke, “Using thematic analysis in psychology,” Qual. Res. Psychol., vol. 3, no. 2, pp. 77–101, 2006.

[13] S. J. Russell et al., Artificial Intelligence: A Modern Approach. New Jersey: Alan Apt, 2021.

[14] Y. K. Dwivedi et al., “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy,” Int. J. Inf. Manage., vol. 71, no. March, 2023, doi: 10.1016/j.ijinfomgt.2023.102642.

[15] A. Sreenivasan and M. Suresh, “International Journal of Innovation Studies Design thinking and artificial intelligence : A systematic literature review exploring synergies,” Int. J. Innov. Stud., vol. 8, no. 3, pp. 297–312, 2024, doi: 10.1016/j.ijis.2024.05.001.

[16] L. Floridi, The Ethics of Artificial Intelligence: Principles, Challenges, and Opportunities, vol. 6649, no. 2023. 2025.

[17] R. Mihalache, “Artificial Intelligence Meets Semiotics : Optimising Advertising Strategies across Cultures,” in Sciendo, 2025, pp. 4652–4670, doi: 10.2478/picbe-2025-0355.