Idea Generation Tool

1. Overview

This was my Masters thesis project at the Tangible Embodied Child-Computer Interaction Lab, School of Interactive Arts and Technology, Simon Fraser University, Canada.

Novice tangible interaction design students often find it challenging to generate input action ideas for tangible interfaces. To address this problem I designed and evaluated a tool consisting of interactive physical artifacts coupled with digital examples of tangible systems and technical implementation guidance. The tool helped in generating input action ideas by enabling to experience input actions, supporting hands-on explorations, and introducing possibilities. However, introducing examples at times caused design fixation.

I published the project at ACM CHI in 2019 and 2021. ACM Conference on Human Factors in Computing Systems (CHI) is a premier peer-reviewed conference for human-computer interaction (HCI). Google Scholar and SJR list ACM CHI as the number 1 publication venue for HCI research with SJR h-index 165. Being the best, it has a low acceptance rate of around 25%.

Duration

January 2018 - May 2020 (2 years 5 months)

Team

Principal Researcher and Designer

Myself

Supervisors

Dr. Alissa N. Antle

Dr. Carman Neustaedter

Team


2. What is Tangible Interaction Design?

Tangible interaction design (TIxD) involves designing tangible user interfaces (TUIs). TUIs involve physical artifacts that represent digital information and serve as interactive controls of the computational media [1]. Such interfaces enhance interaction with computational applications by building upon users’ knowledge and skills of interacting with their physical environments [2,3,4].

Input actions are the physical actions (e.g. rotate, slide, squeeze) performed by the users on the interface to interact with the computational application.

Following is an example of a TUI from the Tangible Media Group, MIT Media Lab.


3. Methodology

I followed the design research methodology [2] to discover, redefine, and investigate the problem by designing and evaluating interventions based on empirical and secondary research data.

Objectives


Tasks and Methods

Secondary research

Literature research

Competitive analysis

Problem framing and scoping


User research

Contextual inquiry

Shadowing

Interviews

Design

Idea generation

Selection matrix

Idea detailing

Wireframing

Evidence-based design

Iterative prototyping

Physical prototyping

Evaluation

Case study

Observation

Expert review

Heuristic markups

Lab study

Pilot study

Interviews

Surveys

Alignment table

Data analysis

Affinity analysis

Open coding

Axial coding

Selective coding

Triangulation

Statistical analysis

Video analysis

Design deliverable analysis


4. Secondary Research


Literature Review

I did a literature review covering in and out of tangible interaction design (TIxD) including theories, frameworks, history, strengths & limitations, application areas, design principles & methods, tools, and education. The literature research also involved four other domains:



Competitive Analysis

I did competitive analysis of tangible tools for idea generation and design tools for tangible interaction design.



Insights from Secondary Research


5. User Research

3

Students

3 male

25-34 years old

Novice students

1

Lecturer

Female

1

Course

Participated in a graduate course

Scoping

I focused on novice tangible interaction design students - students learning tangible interaction design for the first time and with no prior experience of the domain and similar domains that involve embodied interactions (e.g., gestural interfaces).



Objectives

  • Understand how novice tanigble interaction design students generate ideas for input actions.

  • Identify challenges faced by these students in generating ideas for input action.



User Screening

I used convenience sampling to screen a tanigble interaction design lecturer and novice tanigble interaction design students.



Data Collection

  • I did contextual inquiry and participatory shadowing by enrolling in a four months project-based graduate-level tanigble interaction design course (IAT 884) at SIAT, SFU that involved many novice tanigble interaction design students. It gave me a first-hand experience of the course activities along with the opportunity to observe, interact, and discuss with novice tanigble interaction design students as their peer.

  • Individual semi-structured interview with the lecturer of IAT 884, who had around 10 years of teaching experience, involving open-ended questions like -

How satisfied you usually are with the input action ideas generated by your students?

Have you noticed your students facing challenges when generating input action ideas?

  • Individual semi-structured interviews (video recorded) with 3 IAT 884 students, involving open-ended questions like -

Can you tell me how you came up with the TUI idea for your course project?

Why did you choose to incorporate this particular input action in your TUI idea?

Did you face any challenges when generating ideas for input actions?



Data Analysis

I transcribed the interviews and did affinity analysis of the qualitative data.



Insights


6. IdeaBits

I designed a tangible tool, IdeaBits, as an exploratory research instrument to identify potential ways of supporting novice tangible interaction design students to generate input action ideas and things that should be avoided. The video below introduces IdeaBits.

Introducing the first version of IdeaBits.

The graphical user interface of revised IdeaBits.

Design Goal

Help novice tangible interaction design students to generate as many input action ideas as possible (preferably diverse) without worrying about their value.



Features



Design Rationales


7. Design Process


8. Evaluation Methodology


I conducted an exploratory case study [20] to identify ways in which potential support can be provided to novice tangible interaction design students to generate input action ideas and things that should be avoided. I investigated students’ interactions and experience with IdeaBits, while they used it to solve a given design problem.



Research Questions

  • In what ways do users use IdeaBits 2.0 to generate TUI ideas?

  • In what ways do users think IdeaBits 2.0 supports them to generate TUI ideas?

  • What challenges and limitations do users face while using IdeaBits 2.0 to generate TUI ideas?



User Recruiting and Screening

To meet the tight timeline, I quickly recruited participants for the study within 5 days. I designed eye-catching flyers and posters to recruit participants. I put up the posters on the notice boards at the School of Interactive Arts and Technology (SIAT) and on social media platforms. I gave brief presentations of the study at lectures and labs of 8 SIAT undergraduate courses, followed by the distribution of the flyers. I also signed up interested students on the spot by collecting their contact information for follow-up.


Using a screening questionnaire I screened novice tangible interaction design students based on their educational and work experience in tangible interaction design or similar domains involving embodied interactions (e.g. gestural interfaces, virtual reality). I also considered their ability to communicate in English and with no self-identified physical or cognitive disabilities.

After conducting the evaluation with 9 participants, I found that the data was repeating, and there was no new significant data. Hence I stopped the study after running it with 12 participants.



Evaluation Procedure

The evaluation involved one-hour individual design sessions conducted in a lab at the School of Interactive Arts and Technology. I provided participants IdeaBits, including both sets of artifacts so that the participants could use whichever they preferred. I also provided some stationery and modeling materials.

I began the evaluation sessions by introducing participants to IdeaBits and the design task. To avoid participants from getting familiar with the examples in IdeaBits, I introduced the graphical user interface by showing tangible user interface examples of only one input action while encouraging participants to interact with only one artifact from each set. I mentioned that they were expected to use the prototype but were not limited to the introduced input actions and examples.

I then conducted the video recorded 1-hour design session during which the participant was left alone in the room to do the given design task. I remotely observed these sessions from another room by live-streaming the video. I took observation notes such as what they were doing (building prototype, sketching, etc.), what and how they were using (which artifact, modeling material, etc.), and anything unexpected that stood out in relevance to the research questions.

Remote observation using TeamViewer [21] and live-streaming of video recorded design sessions.

Following the design session, I conducted semi-structured interviews (video recorded) to investigate how the participants used IdeaBits to do the design task and their experience. I asked open-ended questions like -

Can you walk me through how you came up with this final idea?

What are the ways you used IdeaBits to do this design task?

In what ways, if any, do you think IdeaBits hindered or limited you in doing the design task?

During the interviews I also followed up on the crucial findings from remote observation by using open-ended questions to probe for reasons behind the observed behaviors. I ended the evaluation session by providing the participants with remuneration of $25 CAD.



Design Task

I generated several ideas for the design task and selected the final one based on several factors such as-



Data Collection

Pilot Study

I iteratively developed the evaluation procedure, design task, and data collection methods discussed above by running a pilot study with 5 participants, following which I evaluated IdeaBits with 12 participants. Some of the insights from the pilot study include -

  • Almost all the pilot participants assumed that they were limited to the input actions introduced by IdeaBits since they were asked to use the tool during the design session. Only some of them asked during the introduction to verify this assumption. Hence to maintain uniformity among the participants and avoid any implicit assumptions, I mentioned during the introduction that they could use any input actions and were not limited to the five introduced possibilities.

  • Some of the pilot participants mentioned that they lost track of time. I added voice reminders to the 1-hour timer [23] at 40, 20, and 5 minutes remaining to help the participants manage their time.

  • I initially placed the two tables in the room parallelly next to each other. The first two pilots mentioned that they found it difficult to shift between the tables. I then put the tables at 90 degrees. This arrangement allowed the pilot participants to move without leaving the chair with casters and hence facilitated switching between the two tables quickly.


9. Data Analysis Process

I collaborated with two peer tangible interaction design researchers (Ofir, and Victor) to analyze the data. We did affinity analysis, open coding, axial coding, selective coding, triangulation, video analysis, and statistical analysis.


Interview Analysis

I and Ofir iteratively and inductively analyzed the interview data using open, axial, and selective coding [24-26].

We individually analyzed two interviews at a time followed by a meeting to discuss our analysis and arrive at a set of mutually agreed-upon themes. Next, we used these themes to individually analyze two more participants. This process was repeated until the set of themes required no more changes. We triangulated different data sources, used detailed descriptions, and presented disconfirming evidence, which contributes to validity.


Participants' Deliverables Analysis

In deliverable 3, the participants listed the emotions, input actions, and outputs in their final idea. I analyzed this data looking for the types of input actions and outputs in the participants’ final ideas while comparing it against 1) input actions and outputs in IdeaBits 2.0 and 2) input actions not in IdeaBits 2.0 that are (not) found commonly in everyday devices and TUIs. In deliverable 4, the participants listed the electronic components required to prototype their final idea. I and Victor analyzed this data for completion, feasibility, errors, and the presence of the listed electronic components in the sensor-enabled artifacts.


Other Data Analysis

We also analyzed the video recording of the design sessions and interviews, participants' deliverables, and remote observation notes to-

  • Clarify and crosscheck the interview transcript data as needed.

  • Look for evidence that supported or was in discrepancy with the themes that emerged from the interview data, while having a secondary focus on any new emerging patterns and findings.


10. Insights from Evaluation

Themes

7 themes emerged from the data analysis.


Below I highlight some of the crucial findings from these themes.



Design Recommendations

Below I highlight some of the design recommendations I framed based on the findings.


11. Conclusion

Contributions


Limitations Future Work


What I Learned

Achievements


  • I published a full paper from the project at ACM CHI 2021 and a short paper with poster presentation at ACM CHI 2019.

ACM Conference on Human Factors in Computing Systems (CHI) is a premier peer-reviewed conference for human-computer interaction (HCI). Google Scholar and SJR list ACM CHI as the number 1 publication venue for HCI research with SJR h-index 177. Being the best, it has a low acceptance rate of around 25%.


  • I secured funding for the project from the Social Sciences and Humanities Research Council of Canada (SSHRC).


  • I successfully defended this project as my thesis to earn my Master's degree in human-computer interaction from Simon Fraser University, Canada.

References

  1. B. Ullmer and H. Ishii. 2000. Emerging frameworks for tangible user interfaces. IBM Systems Journal 39, 3.4: 915–931. https://doi.org/10.1147/sj.393.0915
  2. Hiroshi Ishii. 2007. Tangible user interfaces. Human-Computer Interaction: Design Issues, Solutions, and Applications: 141–157.
  3. Robert JK Jacob, Audrey Girouard, Leanne M Hirshfield, Michael S Horn, Orit Shaer, Erin Treacy Solovey, and Jamie Zigelbaum. 2008. Reality-based interaction: a framework for post-WIMP interfaces. 201–210.
  4. Orit Shaer and Eva Hornecker. 2010. Tangible User Interfaces: Past, Present, and Future Directions. Found. Trends Hum.-Comput. Interact. 3, 1–2: 1–137. https://doi.org/10.1561/1100000026
  5. Smit, D., Oogjes, D., de Rocha, B. G., Trotto, A., Hur, Y., & Hummels, C. (2016, February). Ideating in skills: Developing tools for embodied co-design. In Proceedings of the TEI'16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 78-85).
  6. Jaasma, P., Smit, D., Van Dijk, J., Latcham, T., Trotto, A., & Hummels, C. (2017, March). The blue studio: Designing an interactive environment for embodied multi-stakeholder ideation processes. In Proceedings of the eleventh international conference on tangible, embedded, and embodied interaction (pp. 1-10).
  7. Analytis, S., Sadler, J., & Cutkosky, M. R. (2015). Paper robot: A design activity to increase beginner’s prototyping confidence with microcontrollers.
  8. Mads Vedel Jensen, Jacob Buur, and Tom Djajadiningrat. 2005. Designing the user actions in tangible interaction. 9–18.
  9. Tom Djajadiningrat, Ben Matthews, and Marcelle Stienstra. 2007. Easy doesn’t do it: skill and expression in tangible aesthetics. Personal and Ubiquitous Computing 11, 8: 657–676.
  10. Steve Benford, Holger Schnädelbach, Boriana Koleva, Rob Anastasi, Chris Greenhalgh, Tom Rodden, Jonathan Green, Ahmed Ghali, Tony Pridmore, and Bill Gaver. 2005. Expected, sensed, and desired: A framework for designing sensing-based interaction. ACM Transactions on Computer-Human Interaction (TOCHI) 12, 1: 3–30.
  11. SparkFun Electronics. Retrieved July 16, 2018 from https://www.sparkfun.com/
  12. Uddipana Baishya. Library of input actions in TUIs. Google Docs. Retrieved March 22, 2020 from https://docs.google.com/spreadsheets/d/1mDj8fw7ygOgyKnxygT-7Ej26Z8Vs8yFWVaYuSBi3s68/edit?usp=sharing&usp=embed_facebook
  13. Philipp Schoessler, Sang-won Leigh, Krithika Jagannath, Patrick van Hoof, and Hiroshi Ishii. 2015. Cord UIs: Controlling Devices with Augmented Cables. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’15), 395–398. https://doi.org/10.1145/2677199.2680601
  14. Anita Vogl, Patrick Parzer, Teo Babic, Joanne Leong, Alex Olwal, and Michael Haller. 2017. StretchEBand: Enabling Fabric-based Interactions through Rapid Fabrication of Textile Stretch Sensors. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17, 2617–2627. https://doi.org/10.1145/3025453.3025938
  15. Karin Niemantsverdriet and Maarten Versteeg. 2016. Interactive Jewellery as Memory Cue: Designing a Sound Locket for Individual Reminiscence. In Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’16), 532–538. https://doi.org/10.1145/2839462.2856524
  16. Billy Cheng, Maxine Kim, Henry Lin, Sarah Fung, Zac Bush, and Jinsil Hwaryoung Seo. 2012. Tessella: Interactive Origami Light. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (TEI ’12), 317–318. https://doi.org/10.1145/2148131.2148200
  17. Toshiki Sato, Haruko Mamiya, Hideki Koike, and Kentaro Fukuchi. 2009. PhotoelasticTouch: Transparent Rubbery Tangible Interface Using an LCD and Photoelasticity. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology (UIST ’09), 43–50. https://doi.org/10.1145/1622176.1622185
  18. Yuichiro Katsumoto, Satoru Tokuhisa, and Masa Inakage. 2013. Ninja track: design of electronic toy variable in shape and flexibility. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (TEI ’13), 17–24. https://doi.org/10.1145/2460625.2460628
  19. uddipana. 2020. uddipana/IdeaBits. Retrieved June 14, 2020 from https://github.com/uddipana/IdeaBits
  20. Robert K Yin. 2009. Case study research : design and methods / Robert K. Yin. Los Angeles : Sage Publications.
  21. TeamViewer – Remote Support, Remote Access, Service Desk, Online Collaboration and Meetings. Retrieved September 14, 2019 from https://www.teamviewer.com/en/
  22. Kieran Woodward, Eiman Kanjo, Samuel Burton, and Andreas Oikonomou. 2018. EmoEcho: A Tangible Interface to Convey and Communicate Emotions. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers - UbiComp ’18, 746–749. https://doi.org/10.1145/3267305.3267705
  23. matinlotfali/AndroidTimer. GitHub. Retrieved September 14, 2019 from https://github.com/matinlotfali/AndroidTimer
  24. John W. Creswell and Cheryl N. Poth. 2018. Qualitative inquiry & research design: choosing among five approaches. SAGE, Los Angeles.
  25. William Gibson and Andrew Brown. 2009. Working with Qualitative Data. SAGE, Los Angeles.
  26. Anselm L. Strauss and Juliet M. Corbin. 1998. Basics of qualitative research: techniques and procedures for developing grounded theory. Sage Publications, Thousand Oaks.
  27. Donald A. Schön. 1983. The reflective practitioner: how professionals think in action. Basic Books, New York.
  28. Dorothé Smit, Doenja Oogjes, Bruna Goveia de Rocha, Ambra Trotto, Yeup Hur, and Caroline Hummels. 2016. Ideating in Skills: Developing Tools for Embodied Co-Design. In Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’16), 78–85. https://doi.org/10.1145/2839462.2839497
  29. Amanda Parkes and Hiroshi Ishii. 2010. Bosu: a physical programmable design tool for transformability with soft mechanics. In Proceedings of the 8th ACM Conference on Designing Interactive Systems, 189–198.
  30. Hernan Casakin. 2005. DESIGN AIDED BY VISUAL DISPLAYS: A COGNITIVE APPROACH. Journal of Architectural and Planning Research 22, 3: 250–265.
  31. Philemonne Jaasma, Dorothé Smit, Jelle van Dijk, Thomas Latcham, Ambra Trotto, and Caroline Hummels. 2017. The Blue Studio: Designing an Interactive Environment for Embodied Multi-Stakeholder Ideation Processes. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’17), 1–10. https://doi.org/10.1145/3024969.3025002
  32. Dorothé Smit, Doenja Oogjes, Bruna Goveia de Rocha, Ambra Trotto, Yeup Hur, and Caroline Hummels. 2016. Ideating in Skills: Developing Tools for Embodied Co-Design. In Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’16), 78–85. https://doi.org/10.1145/2839462.2839497
  33. David G. Jansson and Steven M. Smith. 1991. Design fixation. Design Studies 12, 1: 3–11. https://doi.org/10.1016/0142-694X(91)90003-F
  34. A. Terry Purcell and John S. Gero. 1996. Design and other types of fixation. Design Studies 17, 4: 363–383. https://doi.org/10.1016/S0142-694X(96)00023-3
  35. Ian Tseng, Jarrod Moss, Jonathan Cagan, and Kenneth Kotovsky. 2008. The role of timing and analogical similarity in the stimulation of idea generation in design. Design Studies 29, 3: 203–221. https://doi.org/10.1016/j.destud.2008.01.003
  36. Tom Djajadiningrat, Ben Matthews, and Marcelle Stienstra. 2007. Easy doesn’t do it: skill and expression in tangible aesthetics. Personal and Ubiquitous Computing 11, 8: 657–676.
  37. Kaisa Väänänen-Vainio-Mattila, Jani Heikkinen, Ahmed Farooq, Grigori Evreinov, Erno Mäkinen, and Roope Raisamo. 2014. User experience and expectations of haptic feedback in in-car interaction. In Proceedings of the 13th International Conference on Mobile and Ubiquitous Multimedia (MUM ’14), 248–251. https://doi.org/10.1145/2677972.2677996
  38. Henning Pohl, Andreea Muresan, and Kasper Hornbæk. 2019. Charting Subtle Interaction in the HCI Literature. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), 1–15. https://doi.org/10.1145/3290605.3300648
  39. Martin Jonsson, Anna Ståhl, Johanna Mercurio, Anna Karlsson, Naveen Ramani, and Kristina Höök. 2016. The Aesthetics of Heat: Guiding Awareness with Thermal Stimuli. In Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’16), 109–117. https://doi.org/10.1145/2839462.2839487
  40. Wang, M., Lei, K., Li, Z., Mi, H., & Xu, Y. (2018, March). Twistblocks: Pluggable and twistable modular tui for armature interaction in 3d design. In Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 19-26).
  41. Huang, Y., & Eisenberg, M. (2012, February). Easigami: virtual creation by physical folding. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (pp. 41-48).
  42. Huang, Y., & Eisenberg, M. (2012, February). Easigami: virtual creation by physical folding. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (pp. 41-48).
  43. CHI Conference - Home. Retrieved June 10, 2020 from https://dl.acm.org/conference/chi
  44. TEI Conference - Home. Retrieved June 10, 2020 from https://dl.acm.org/conference/tei

More Projects