Now you see it! Using wearable cameras to gain insights into the lived experience of cardiovascular conditions 

Abstract

Wearable cameras offer an innovative way to discover new insights into the lived experience of people with cardiovascular conditions. Wearable cameras can be used alone or supplement more traditional research methods, such as interviews and participant observations. This paper provides an overview of the benefits of using wearable cameras for data collection and outlines some key considerations for researchers and clinicians interested in this method. We provide a case study describing a study design using wearable cameras and how the data were used.

The problem: understanding the lived experience of people with cardiovascular conditions

Health researchers and clinicians seek to understand the lived experience of people with cardiovascular conditions, including factors that influence health behaviours. Knowledge of lived experiences may be used to develop theories, inform the design of new interventions, or provide an evaluation of an intervention. In the European Journal of Cardiovascular Nursing, examples of studies that sought to further our understanding of this topic have used a range of methodologies, including brief assessment scales,¹ qualitative interviews,² and observational studies.³ These traditional methods rely on participants to remember and accurately portray their past experiences, which can be subject to participant and researcher biases that may affect research findings.⁴

Questionnaires, such as the European Heart Failure Self-Care Behaviour Scale and the Self-Care of Heart Failure Index,^5,6 can provide a snapshot of a person’s health status or behaviours. They can also be scaled to obtain data on large numbers of participants. However, the output of this type of research usually lacks contextual information, and may not illustrate the complexities of human behaviour. While qualitative interviews provide rich, in-depth information, the quality of these data relies upon the interviewer's skills and candidness of the interviewee, and may be subject to recall bias. Ethnographic methods, such as participant observation, do not rely on participants’ ability or willingness to communicate their experiences. However, ethnographic observations are time-consuming, and resource-intensive for the researcher and may be disruptive to the regular routines of the participants, again leading to findings that may be different from their everyday experiences.

As health systems focus more on personalized care and provide individuals with tailored interventions and support, our research methods should follow suit. Technologies such as wearable cameras allow us to supplement existing research methods to provide insight into the wearer’s unique experiences, overcoming some of the limitations of traditional methods. This article aims to provide researchers and clinicians with an understanding of the application of wearable cameras in cardiovascular research.

A solution: wearable cameras to provide rich contextual data and prompt memory

Wearable cameras are a technology that can capture the experiences and environments of the wearer digitally.⁷ This technology emerged from the research domain called lifelogging, in which individuals seek to capture life activities using wearable and informational sensors.⁸

Wearable cameras are small and lightweight, usually worn around the neck or clipped onto clothing. Such wearable cameras capture images or video automatically (without user input) at various time intervals (e.g. every 30 s), and many can also be triggered to capture an image by a button press (as with a conventional camera). Some wearable cameras are equipped with sensors such as microphones and accelerometers to capture additional data. People often forget they are wearing a camera. These features of wearable cameras mean they do not change the user’s daily activities,⁹ offering a strong rationale for their use in understanding part of the lived experience of people with chronic health conditions.

A scoping review conducted by our team found that wearable camera data have been used to supplement traditional data collection methods with the aim of capturing a range of health-related behaviours and risk factors for chronic conditions.¹⁰Central illustration provides examples of how wearable cameras might augment or replace other research methods, including photo-elicitation.¹¹ To date, no study has reported any serious adverse events, including those related to participant or bystander privacy.¹² However, in line with existing research that has developed an ethical framework and guidelines for using wearable cameras in research,^12,13 we recommend that precautions to promote autonomy and reduce risks are considered when designing and conducting a study. Implicit in our recommendations for hardware and software are the key considerations from these frameworks.

Currently, there are no formal guidelines for the analysis of wearable camera images, but principles may be taken from qualitative research methods, including visual ethnography,¹⁴ photo-elicitation,¹¹ and autophotography.¹⁵ Furthermore, sophisticated technologies, such as computer vision tasks that (semi)-automatically recognize and classify images, allow researchers to analyse large data sets in a fraction of the time it would take to sort and review images manually (see case study example).

Hardware and software

There is a range of wearable cameras, including some that are/were commercially available such as Microsoft’s Sensecam and GoPro’s action cameras. Consistent with existing frameworks on the ethical and safe use of wearable cameras,^12,13 researchers may wish to consider the following factors when choosing a wearable camera for their research:

• Cost—What is the research budget for equipment and data analysis? Cameras range in price from tens to hundreds of dollars each. Automated analysis is likely to be significantly less costly than manual analysis. As of 2022, the typical cost of automatically ‘coding’ an image using state of the art programming is approximately 0.1–0.2 cents per image. While it is difficult to provide a direct cost comparison of manual annotation, typically, it would take a person 10–60 s to review one image which is more time-consuming and thus costly than automated analysis.

• Quality of images—How detailed do images need to be to meet the needs of the study? Will the camera capture images with sufficient detail to facilitate later analysis? Image size, camera lens angle, and low-light performance are important considerations.

• Data capture—How frequently can images be captured and how (e.g. every 30 s)? Naturally, there is a trade-off between battery life and frequency of capture. Many modern wearable cameras (e.g. the Narrative clip) capture images in an encrypted manner, which cannot be accessed without the correct software. Usually, images from these cameras are downloaded via a USB connection to a computer. This can be initiated by either the participant or the researcher. Other cameras use software to automatically transfer the images to a computer and then remove images from the camera. After download, images can be stored on a local computer, network server, or secure cloud computing infrastructure, depending on requirements.

• Size and weight of camera—Will the participant be concerned about the camera's weight or impacted by discomfort when wearing it? Most cameras are lightweight, but even these can drag down clothing if attached via a clip.

• Ease of use of the camera—Will participants be able to recharge and use the camera in daily life? Most cameras need to be connected to a computer to upload content and recharge.

• Privacy functions—Will the participant be easily able to turn the camera on/off in different situations (e.g. in crowded places, in the bank, at work, while carrying out personal hygiene activities), or will some content need to be deleted post-capture? Off-the-shelf software tools can locate and redact all faces present in images, as well as other content such as readable text. Deep-learning artificial intelligence techniques will allow for the further removal of any objects deemed private to the participant.

• Battery life—How long will the cameras need to be worn, when will participants need to charge the device (e.g. overnight) and how regularly? The device must work for typical awake hours, but not all devices will operate for long durations without charging.

• Data storage—Where will data be stored, how much disk space will be needed, and in what format? We typically see 500 MB-1 GB per day being captured as image sequences. If stored on a cloud-hosting service, this can be expensive as the data quantity grows. Whether the data are stored locally or in the cloud, the data should be encrypted to ensure security and privacy preservation.

• Processing—How will data be processed? Wearable cameras can capture a sequence of images but will not inspect them to extract valuable metadata, such as human activity data. Third-party solutions are currently needed for this task.

Case study: The ‘Seeing is Believing’ study

The ‘Seeing is Believing’ study was a prospective observational pilot study that aimed to test the feasibility and utility of wearable cameras to identify self-management practices for people living with heart failure.¹⁶ Additionally, we wanted to determine if these images could enhance self-management. Outpatients diagnosed with heart failure [New York Heart Association (NYHA), functional Class II or III], over 18 years, able to read and understand English, treated with medication, and provided written informed consent were included. The study ran from June 2016 to May 2017 and recruited 30 participants. The Narrative Clip camera was worn around the neck during waking hours for 30 days and charged overnight. Figure 1 shows an image of this camera. The camera automatically took an image in the forward-facing direction every 30 s. Participants were instructed to carry out normal activities during the study period; no advice to modify lifestyle behaviour was given. Participants were visited by a researcher twice a week; during this visit, images were manually downloaded, and participants were allowed to review and delete images they did not want to share. The images were saved on a secure server. Physiological parameters (e.g. blood pressure, heart rate) were also measured twice weekly, and participants completed a semi-structured acceptability and feasibility questionnaire at study conclusion.

A total of 629 603 images were available for analysis. All participants agreed that the camera was easy to use. Most participants felt comfortable wearing the camera (93%) and thought this technique could be used to help develop interventions to support people with heart failure in the future (93%).¹⁶ This study provided a rich, contextual data set, and our group has used this in several ways.

Example 1: informing the development of an intervention

Co-design is a methodology that allows people to contribute their lived experience to design patient-centred interventions.¹⁷ We are using co-design to develop an intervention to support self-care among people with heart failure. Images from the wearable camera study were used to inform an interview guide that aimed to empathize with people with heart failure. We used maximum variation purposive sampling, a sampling method that aims to provide a diverse range of cases (images) relevant to a topic,¹⁸ to select images for coding. We reviewed images from each participant and selected relevant, unique images that were visually clear. These selected images were then annotated using the categories: environment, objects, activities, and people. The image descriptions were used to challenge assumptions about self-care among people with heart failure, and an interview guide was developed to explore different aspects of self-care in more detail, further testing these assumptions. Figure 2 provides a worked example of this process. Additionally, personas were developed to represent people with heart failure using data from the interviews and existing literature. Personas are representations of a population group. Images from the ‘Seeing is Believing’ study were used to supplement the personas, providing the research team and developers with a deeper insight into the people they were designing the intervention with.

Example 2: using automated techniques to categorize the images

The value of wearable cameras lies in the automated and passive capture of large volumes of data about the life activities of the participant. However, manual analysis of hundreds or thousands of images per day, per participant can take a significant amount of time—as we learned with our ‘Seeing is Believing’ study. Consequently, we have explored different options for automatic and semi-automatic annotation of the images.

Recent advances in computer vision deep-learning tools can now actively learn to automatically annotate, categorize and even describe any visual image as text. Given sufficient training data (annotations from humans), such deep-learning algorithms can be applied to wearable camera images to better understand patterns of self-management behaviour. Deep-learning algorithms, such as convolutional neural networks (CNNs),¹⁹ are widely used in computer vision tasks, such as medical image analysis, and have been shown to achieve better results than human experts’ average performance. They have been used to detect diabetic retinopathy,²⁰ skin lesions,²¹ colon cancer,²² and anomalies of the heart²³ and breast.²⁴ The CNNs can be trained on any form of image data, so they can also be trained on wearable camera images to determine patterns of self-management behaviour in people with heart failure and to predict hospital admissions. Hence, it is proposed that wearable cameras with CNNs for behaviour identification hold the potential to understand patterns of self-management at a low cost and scale.

Example 3: future application—a nurse-led intervention

Wearable cameras also offer potential for augmenting care of people with cardiovascular conditions, such as heart failure. This condition has a variable clinical course and frequent exacerbations of symptoms; hence, appropriate self-care is critical to maximize treatment benefits.²⁵ To maximize the impact of nurse-led support for heart failure, nurses must understand each individual’s challenges. Improved automated analysis, with increased choice in how large amounts of images are categorized, sorted, and displayed, allows flexibility for images to be used in clinical practice. With this type of analysis, a targeted image review of a particular self-management category (e.g. daily routines, medication management, diet and fluid intake, physical activity, and social support) or a particular time of day (e.g. breakfast) can now be easily identified.

For example, a newly diagnosed heart failure patient could wear a camera at home in the initial weeks following discharge from hospital. Images could then be reviewed by the nurse, patient, and family members, to understand the person’s lived experience and tailor feedback and (self)-management plans. Additionally, a person experiencing an exacerbation of heart failure could use wearable camera images to inform an updated self-management plan or reveal self-management behaviours that may be exacerbating their condition. There are many other chronic conditions that wearable camera images could assist, such as chronic respiratory conditions and diabetes.

Reporting

The research and clinical community have raised questions regarding the use of wearable cameras; thus, studies using this method must be clearly reported. Informed by our experience in using wearable cameras, ethical guidelines, and qualitative research guidelines, we suggest consideration of the following when reporting studies using wearable cameras:

• Title/abstract: identify the use of wearable cameras in the study

• Rationale: a clear justification for using wearable cameras in addition to, or instead of other data collection methods

• Researcher characteristics: information on researcher characteristics (e.g. experience) that could influence the research (e.g. when undertaking a manual analysis)

• Sampling strategy: information about the sampling of participants and images (e.g. images selected from specific dates/events or selected at random) and a justification of these choices

• Ethical issues: information on adherence to ethical guidelines, including detail and reasons for adaptations, or disregard of guidelines

• Data collection methods: description of procedures, including instructions and training provided to participants and dates of data collection

• Data collection tools: details about the camera (e.g. brand/model, frequency of image capture, privacy functions, the intended location of wear)

• Data management: information on data storage and data processing (e.g. deletion of images by participant or researcher, anonymization of images, software to support analysis, and reporting)

• Data analysis

  • Manual: information about coding, frameworks or theories, and software/tools used (e.g. NVivo)

  • Automated: software used for coding and settings, data analysis approaches (e.g. deep-learning algorithms, validation metrics)

• Results: detail on participants, number of images collected and included in the study, and participation (including reasons for dropout). Consider presenting a sample of images or worked examples

• Discussion: as well as the main findings, provide information on the strengths and limitations of wearable cameras to meet the study aim

Strengths and limitations of wearable cameras

In this study, we have explored the usefulness of wearable cameras as a research method. Wearable cameras can facilitate a rich and complementary understanding of the lived experience of cardiovascular conditions without influencing behaviour, as may be the case in in-person observation or reporting bias that exists in questionnaires and interviews. They can also provide insights that may challenge existing judgements, opening researchers, clinicians, and broader stakeholders to new patterns and possibilities, informing future practice. Additionally, depending on the available resources and methods used for data analysis, they are scalable, potentially providing information about a wide range of people and experiences. However, as highlighted in this study, wearable cameras are not without limitations. Firstly, wearable cameras produce a vast quantity of data, so careful consideration of data collection time frames (e.g. days, weeks, and months) and how to approach analysis (manual or automated) is required, especially concerning cost. Some automated analysis tools are not yet precise enough to detect some activities. To meet this level of precision, large data sets are required to train machine learning models. Short-lived activities (e.g. taking medication) can be missed as they occur ‘between’ image capture. As a result, wearable cameras alone may not be sufficient to meet the research aims. In this case, other tools can supplement data from wearable cameras (e.g. sensors, interviews, and diaries). The security of data from wearable cameras is also concern; images often contain personal data, so care must be taken to ensure that data are stored securely (e.g. by encrypting the data) regardless of where they are stored. Finally, there are concerns around privacy for the participant and bystanders, but software can be used to redact image content such as faces or text. Understanding the strengths and limitations of wearable cameras will help researchers navigate the design of studies using this innovative method.

Conclusion

Insights on the lived experience of cardiovascular conditions are important to researchers and clinicians. Traditional research methods have limitations that may be overcome by using wearable cameras. As wearable camera technology improves and the accuracy of trained models (e.g. CNNs) is enhanced, wearable cameras will likely become an important tool for health researchers. We recommend that researchers and clinicians consider using wearable cameras to evolve the evidence based on people's lived experience with cardiovascular conditions. Furthermore, wearable cameras may be applied to clinical interventions that seek to provide personalized advice and care to people living with or at risk of cardiovascular conditions.

Acknowledgements

The authors would like to thank the participants of the 'Seeing is Believing' study for their time and participation.

Funding

R.N. is supported by a Deakin University Postgraduate Research Scholarship. S.C. is supported by a National Heart Foundation of Australia postdoctoral fellowship (#104860). The ‘Seeing is Believing’ study was funded by a National Heart Foundation Vanguard Grant, Australia (#101348).

Data Availability

The data set described in this study cannot be shared publicly due to the privacy of individuals participating in the study.

References

Author notes