A System to Support Memory Recall by Creating Tag Clouds from Calendars, Twitter, and Photos
Abstract
Human memory is crucial for many daily activities. However, human memory is unreliable.
Since existing memory aids such as diaries and notebooks need some effort for writing and accessing, we present an approach to capture and display memory cues from the user’s daily activities, such as writing schedules, posting SNS messages, and taking photos. We developed a system that supports memory recall by creating tag clouds from the user’s calendars and Twitter posts. The main feature of this research is using the number of photos taken by users to weight the tags to recall impressive events. This is based on the hypothesis that impressive events are found on the days when users took many photos. We evaluated out tag cloud generation algorithms and our implemented prototype system and obtained the following results for user memory recall: (1) Our tag cloud (using our weighting method with calendar and Twitter data) is superior to other tag clouds. (2) Sorting by time in the tag cloud outperforms other sorting methods. (3) Our implemented prototype system is useful.
References
M. Harvey, M. Langheinrich, and G. Ward, “Remembering through lifelogging: A survey of human memory augmentation,” Pervasive and Mobile Computing, vol. 27, pp. 14–26, 2016, doi: 10.1016/j.pmcj.2015.12.002.
H. Murakami, “History structure for exploring desktop data,” in Proc. SIGIR 2010 Workshop on Desktop Search (Understanding, Supporting and Evaluating Personal Data Search), 2010, pp. 25–26.
A. W. Rivadeneira, D. M. Gruen, M. J. Muller, and D. R. Millen, “Getting our head in the clouds: Toward evaluation studies of tagclouds,” in Proc. SIGCHI Conf. Human Factors in Computing Systems (CHI ’07), 2007, pp. 995–998, doi: 10.1145/1240624.1240775.
M. J. Halvey and M. T. Keane, “An assessment of tag presentation techniques,” in Proc. 16th Int’l Conf. World Wide Web (WWW ’07), 2007, pp. 1313–1314, doi: 10.1145/1242572.1242826.
D. Helic, C. Trattner, M. Strohmaier, and K. Andrews, “Are tag clouds useful for navigation? A network-theoretic analysis,” Int’l Journal of Social Computing and Cyber Pyhsical Systems, vol. 1, no. 1, pp. 33–55, 2011, doi: 10.1504/IJSCCPS.2011.043603.
M. Matsumoto, “Supporting human recollection of the impressive days using the number of photos,” Master’s Thesis, Graduate School for Creative Cities, Osaka City University (in Japanese), 2014.
M. Matsumoto, S. Matsuura, K. Mitsuhashi, and H. Murakami, “Supporting human recollection of the impressive events using the number of photos,” in Proc. 6th Int’l Conf. Agents and Artificial Intelligence - Volume 1:ICAART, 2014, pp. 538–543, doi: 10.5220/0004902805380543.
P. Venetis, G. Koutrika, and H. Garcia-Molina, “On the selection of tags for tag clouds,” in Proc. 4th ACM Int’l Conf. Web Search and Data Mining (WSDM ’11), 2011, pp. 835–844, doi: 10.1145/1935826.1935855.
H. Murakami and T. Hirata, “Information acquisition and reorganization from the WWW by using Memory-Organizer,” Bulletin of Osaka City University Media Center, vol. 3, pp. 9–14, 2002.
H. Murakami and T. Hirata, “A system for generating user’s chronological interest space from web browsing history,” Int’l Journal of Knowledge-based and Intelligent Engineering Systems, vol. 8, no. 3, pp. 149–160, 2004, doi: 10.3233/KES-2004-8303.
H. Murakami, K. Mitsuhashi, and K. Senba, “Creating user’s knowledge space from various information usages to support human recollection,” in Advanced Research in Applied Artificial Intelligence (IEA/AIE 2012), H. Jiang,W. Ding, M. Ali, and X.Wu, Eds., 2012, pp. 596–605, doi: 10.1007/978-3-642-31087-4 61.
H. Murakami and R. Murakami, “A system using tag cloud for recalling personal memories,” in Advances and Trends in Artificial Intelligence. From Theory to Practice (IEA/AIE 2019), F. Wotawa, G. Friedrich, I. Pill, R. Koitz-Hristov, and M. Ali, Eds., 2019, pp. 398–405, doi: 10.1007/978-3-030-22999-3 35.
H.-H. Huang, Y. Sumi, and T. Nishida, “Personal image repositories as externalized memory spaces,” Int’l Journal of Knowledge-based and Intelligent Engineering Systems, vol. 10, no. 2, pp. 169–180, 2006, doi: 10.3233/KES-2006-10206.
Y. Chen and G. J. F. Jones, “Augmenting human memory using personal lifelogs,” in Proc. 1st Augmented Human Int’l Conf. (AH ’10), 2010, paper 24, doi: 10.1145/1785455.1785479.
A. Mathur et al., “Lifeview: A lifelog visualization tool for supporting sentimental recall and sharing,” in Proc. 24th Australian Computer-Human Int’l Conf. (OzCHI ’12), 2012, pp. 371–380, doi: 10.1145/2414536.2414596.
A. Aiordachioae and R.-D. Vatavu, “Life-tags: A smartglasses-based system for recording and abstracting life with tag clouds,” in Proc. ACM Hum.-Comput. Interact., vol. 3, no. EICS, 2019, paper 15, doi: 10.1145/3331157.
V. Bush, “As we may think,” The Atlantic Monthly, vol. 176, no. 1, pp. 101–108, 1945.
A. J. Sellen et al., “Do life-logging technologies support memory for the past? an experimental study using sensecam,” in Proc. SIGCHI Conf. Human Factors in Computing Systems (CHI ’07), 2007, pp. 81–90, doi: 10.1145/1240624.1240636.
A. J. Sellen and S. Whittaker, “Beyond total capture: A constructive critique of lifelogging,” Commun. ACM, vol. 53, no. 5, pp. 70–77, 2010, doi: 10.1145/1735223.1735243.
J. Gemmell, G. Bell, R. Lueder, S. Drucker, and C. Wong, “Mylifebits: Fulfilling the memex vision,” in Proc. 10th ACM Int’l Conf. Multimedia (MULTIMEDIA ’02), 2002, pp. 235–238, doi: 10.1145/641007.641053.
“NTCIR16 lifelog task.” [Online]. Available: http://ntcir-lifelog.computing.dcu.ie/
C. Gurrin et al., “Experiments in lifelog organisation and retrieval at NTCIR,” in Evaluating Information Retrieval and Access Tasks, T. Sakai, D. Oard, and N. Kando, Eds., 2021, pp. 187–204, doi: 10.1007/978-981-15-5554-1 13.
C. Gurrin et al., “Introduction to the third annual lifelog search challenge (LSC’20),” in ICMR ’20: Proc. 2020 Int’l Conf. Multimedia Retrieval, 2020, pp. 584–585, doi: 10.1145/3372278.3388043.
W. Jones, “Personal information management,” Annual Review of Information Science and Technology, vol. 41, no. 1, pp. 453–504, 2007, doi: 10.1002/aris.2007.1440410117.
W. Jones, The future of personal information management, part I: our information, always and forever. Morgan & Claypool Publishers, 2012, doi: 10.2200/S00411ED1V01Y201203ICR021.
J. Aires and D. Gonc¸alves, “Personal information dashboard - me, at a glance,” in PIM workshop 2012, 2012.
M. Yoo, W. Odom, and A. Berger, “Understanding everyday experiences of reminiscence for people with blindness: Practices, tensions and probing new design possibilities,” in Proc. 2021 CHI Conf. Human Factors in Computing Systems (CHI ’21), 2021, paper 167, doi: 10.1145/3411764.3445212.
S. T. Peesapati et al., “Pensieve: Supporting everyday reminiscence,” in Proc. SIGCHI Conf. Human Factors in Computing Systems (CHI ’10), 2010, pp. 2027–036, doi: 10.1145/1753326.1753635.
D. Cosley et al., “Using technologies to support reminiscence,” in Proc. 23rd British HCI Group Annual Conf. People and Computers: Celebrating People and Technology (BCS-HCI ’09), 2009, pp. 480–484.
U. Torres-Parejo, J. R. Campa˜na, M. A. Vila, and M. Delgado, “A survey of tag clouds as tools for information retrieval and content representation,” Information Visualization, vol. 20, no. 1, pp. 83–97, 2021, doi: 10.1177/1473871620966638.
D. Skoutas and M. Alrifai, “Tag clouds revisited,” in Proc. 20th ACM Int’l Conf. Information and Knowledge Management (CIKM ’11), 2011, pp. 221–230, doi: 10.1145/2063576.2063613.
W. Zhao et al., “Topical keyphrase extraction from Twitter,” in Proc. 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (HLT ’11), 2011, pp. 379–388.
S. Lohmann, J. Ziegler, and L. Tetzlaff, “Comparison of tag cloud layouts: Taskrelated performance and visual exploration,” in Human-Computer Interaction - INTERACT 2009, T. Gross et al., Eds., 2009, doi: 10.1007/978-3-642-03655-2 43.
