The TESS Triple-9 Catalog II: a new set of 999 uniformly vetted exoplanet candidates

ABSTRACT

The Transiting Exoplanet Survey Satellite (TESS) mission is providing the scientific community with millions of light curves of stars spread across the whole sky. Since 2018, the telescope has detected thousands of planet candidates that need to be meticulously scrutinized before being considered amenable targets for follow-up programs. We present the second catalog of the Planet Patrol citizen science project containing 999 uniformly vetted exoplanet candidates within the TESS ExoFOP archive. The catalog was produced by fully exploiting the power of the Citizen Science Planet Patrol project. We vetted TESS Objects of Interest (TOIs) based on the results of Discovery And Vetting of Exoplanets (dave) pipeline. We also implemented the automatic disposition generator, a custom procedure aimed at generating the final classification for each TOI that was vetted by at least three vetters. The majority of the candidates in our catalog, 752 TOIs, passed the vetting process and were labelled as planet candidates. We ruled out 142 candidates as false positives and flagged 105 as potential false positives. Our final dispositions and comments for all the planet candidates are provided as a publicly available supplementary table.

1 INTRODUCTION

Over the years, more than 5000 exoplanets have been discovered as a result of a series of ground- and space-based exoplanet-hunting missions, with thousands more still awaiting confirmation¹. The development of extensive catalogs of confirmed exoplanets is a fundamental step to shed light on the planetary formation processes and provide clues as to whether the Solar system is unique in the Galaxy (e.g. Bach-Møller & Jørgensen 2021). Among the different exoplanet discovery techniques, the transit method has proven to be the most fruitful, leading to the discovery of |$\sim 77~{{\ \rm per\ cent}}$| of all currently known exoplanets¹. In fact, the Kepler mission (Borucki et al. 2010) alone has found over 2700 transit-like signals that were later verified as true planets along with many candidates still waiting further examination (Lissauer et al. 2022). Since its launch in 2018, the Transiting Exoplanet Survey Satellite (TESS; Ricker et al. 2015) space-based mission has observed |$\sim 85~{{\ \rm per\ cent}}$| of the entire sky, collecting the light curves of ∼ 200 000 pre-selected stars at 2-min cadence. Furthermore, TESS also acquired a series of full frame images (FFIs) at 10- and 30-min cadences, with the goal of expanding the transit search to the entire sky; since September 2022 (the mission’s second extension), FFIs have being acquired at a cadence of 200 s. At the time of writing, TESS has detected almost 6000 TESS Objects of Interest (TOIs), and ∼10 000 are expected to be found in the FFIs within the primary mission duration (Barclay, Pepper & Quintana 2018). According to the ExoFOP-TESS archive², in December 2022, 277 candidates out of the currently known 5887 TOIs were validated to date by follow-up measurements.

Detecting a transit-like signal in the light curve of a distant star is not sufficient to confirm the discovery of an exoplanet. Several astrophysical sources (e.g. eclipsing binary stars, stellar spots, and/or pulsations; Ciardi et al. 2018) or instrumental artefacts (e.g. jitter noise and momentum dumps) can mimic a transit-like signal in the light curve of the observed target, leading to a false positive (FP) detection. In light of the many potential false-positive scenarios that affect the photometric transit method, a planet candidate has to be carefully examined before being promoted as a suitable target for spectroscopic follow-up observations aimed at its confirmation as a bona-fide planet. Precision radial velocity (PRV; Baranne et al. 1996; Pepe et al. 2004) measurements are challenging, time-consuming, and achievable by only a handful of instruments.

The vetting procedure is one of the key steps in the process of confirming the planetary origin of a transit feature found in the light curve of a star. A catalog of uniformly vetted transiting planet candidates is essential to optimize spectroscopic follow-up observations by promoting targets for which common false-positive scenarios have been already ruled out. Moreover, the vetting procedure enables statistical validation of planet candidates for which no PRV measurements are feasible. Finally, complementary human vetting also provides the opportunity to create a knowledge base for machine-learning approaches aiming to automate the entire vetting process.

Several automated vetting pipelines have been developed over the years to tackle the issue of FPs in transit photometry. For example, the AUTOVETTER (McCauliff et al. 2015), ROBOVETTER (Coughlin et al. 2016), and SIDRA (Mislis et al. 2016) pipelines are decision-tree based machine-learning codes trained on massive human-inspected data sets to produce uniformly vetted catalogs of planet candidates discovered from the Kepler mission.

Deep learning (DL) algorithms have been trained to identify planet candidates in both Kepler and TESS light curves. These work both as likelihood-based rankers (Shallue & Vanderburg 2018) or binary classifiers (e.g. Olmschenk et al. 2021). Since the innovative and high-performance approach provided by these models, vetting efforts have shifted towards DL methods. Despite the fact that DL models usually outperform traditional machine-learning methods, they come with certain drawbacks. Most notably, DL models are computationally expensive, and the results they produce are sometimes difficult to interpret (Samek, Wiegand & Müller 2017).

Apart from models based on neural networks, pipelines such as VESPA (Morton 2012) and TRICERATOPS (Giacalone et al. 2021) evaluate the Bayesian probability that a signal is a FP based on the shape of the light curve as well as the stellar parameters of the nearby sources within the aperture mask used to extract the light curve. Furthermore, once a certain false-positive threshold value is set, these algorithms allow to statistically validate a signal as a true planet. The Discovery And Vetting of Exoplanets (dave) Kostov et al. (2019a) vetting pipeline determines whether a transit-like signal is caused by a planet candidate or is a FP by testing the candidate at both the pixel and light-curve levels. Building upon methods used for vetting exoplanet candidates from the Kepler mission, dave was designed to analyse transit photometry from the K2 mission and later modified to work with TESS light curves as well (Kostov et al. 2019b; Gilbert et al. 2020).

It is important to note that none of these pipelines can completely replace visual human inspection. Automatic pipelines, for example, can fail to correctly classify signals with low-signal-to-noise ratio (SNR; e.g. small planets with long periods), that are dominated by stellar variability, or that are plagued by various systematic effects and instrumental artefacts. Furthermore, different planet search and/or vetting pipelines use different methods to extract and process the raw data. For example, Kostov et al. (2019a) demonstrated that nearly one in every three K2 planet candidates has insufficient SNR across all available light-curve sets to provide a reliable classification (i.e. planet candidate or FP). Hence, all automated vetting pipelines come with inherent data-processing and data-analysing biases and peculiarities, making complementary human inspection not only recommended but also essential.

Traditionally, complementary human vetting is typically done by a small group of professional astronomers. However, the ever-increasing number of exoplanet candidates in need of careful examination makes this approach impractical. Vetting hundreds of targets by a handful of scientists may take months; unforeseen biases may emerge unless a clear workflow is defined within the team at the start of the work, and strictly adhered to (e.g. Thompson et al. 2018), and an intuitive, interactive, user-friendly vetting platform is used by all vetters.

Citizen science is a powerful way of doing science that is becoming increasingly popular due to new collaboration tools. It offers the opportunity to address the human vetting bottleneck by harnessing the expertise and enthusiasm of amateur astronomers. For example projects like Planet Patrol (Kostov et al. 2022), Planet Hunter TESS (PHT; Eisner et al. 2020), Exoplanet Explorers (Christiansen et al. 2018), and Disk Detective (Kuchner et al. 2016), hosted by the Zooniverse platform (Lintott et al. 2008), helped scientists achieve, in a few weeks, results that would have otherwise taken years to complete.

Planet Patrol is a citizen science project designed to assist with the vetting workflow of TESS planet candidates based on the automated results and dispositions produced by the dave pipeline Kostov et al. (2022). After the first stage of the project was completed on Zooniverse, several citizen scientists expressed interest in continuing assisting the scientific core team with the vetting efforts. Under the guidance of members from our core science team, these ‘superuser’ volunteers were trained to classify TESS planet candidates by critically interpreting and analysing the entire output from dave. The superusers became an integral part of the team and played an essential role in our first TESS Triple 9 Catalog (Cacciapuoti et al. 2022, Paper I hereafter), where they assisted with the vetting of 999 TOIs, classifying 709 of them as planet candidates.

In this work, we present the continuation of our vetting efforts, in the form of a catalog of 999 uniformly vetted TESS planet candidates detected by the Science Processing Operations Centre pipeline (SPOC; Jenkins et al. 2016) and Quick Look Pipeline (QLP; Huang et al. 2020) pipelines. We utilize the same workflow as used in Paper I and introduce several new vetting tools and diagnostics.

The outline of the paper is the following: in Section 2, we discuss the workflow adopted to uniformly vet 999 candidates within the TESS data base, including the new implementations with respect to Paper I. In Section 3, we highlight the details of the Planet Patrol Project and how it helped in carrying out this work. The catalog and its details are discussed in Section 4. Finally, we summarize our conclusions in Section 6.

2 METHOD

We have conducted a uniform vetting of 999 TOIs by means of the dave pipeline. dave utilizes a two-step vetting process for each TESS sector where the target has been observed, namely a pixel-level photocentre analysis and a flux-based analysis at the light-curve level. The pipeline vets both the SPOC short-cadence and the FFI long-cadence TESS data, using the ‘Corrected Flux’ eleanor light curves (Feinstein et al. 2019) for the latter as-is, i.e. without further detrending or post-processing. dave uses the target’s TIC ID, transit ephemeris, depth, and duration as provided by the publicly available ExoFOP website. For completeness, we outline the main products of dave below; for further details, we refer the reader to Kostov et al. (2019a).

• The centroids module generates a difference image by subtracting the overall in-transit image from the corresponding out-of-transit image for each transit and for each sector. Then, for each transit, the code calculates the photocentre of the light distribution by fitting to the difference image the TESS Pixel Response Function and a Gaussian point-spread function (PSF). Finally, the overall position of the photocentre for a particular sector is computed by taking the average over all the transit events detected in that sector. We note that the centroid difference images created by dave can be difficult to interpret when the SNR is low or there are significant artefacts. In such cases, the centroid measurements can be unreliable (flagged as ‘UC’, for ‘unreliable centroid’ in our catalog) and the corresponding automated photocentre disposition might be incorrect. For example, if some of the individual difference images exhibit prominent systematics, the calculated average photocentre position may be affected to the point of dave flagging the candidate as a FP due to a nominal centroid offset (CO). Thus, it is important for a human vetter to inspect the individual difference images, the corresponding photocentre measurements, and the average difference image, as well as evaluate the reliability of the automated photocentre dispositions provided by dave. The vetter is trained to distinguish between valid difference images and photocentre measurements, and those that could be affected by instrumental and/or computational systematics. The vetter ignores the poor measurements and makes a final decision based on the reliable photocentres. For example, if there is a clear CO with respect to the catalog position of the target star, and there are no obvious systematics that might affect the measurements, then the candidate is flagged as an FP (see Fig. 1).

• The Modelshift module uses the phase-folded light curve along with the best-fitting trapezoid transit model to evaluate the significance of the primary signal together with any secondary and tertiary signals, as well as of potential odd-even difference (OED) between consecutive transits. This module determines whether the source of the signal is consistent with an eclipsing binary system instead of a transiting planet. For example, if there is a significant secondary eclipse at any phase other than zero or an OED, dave flags the target as an FP (see Fig. 2). We note that since dave uses the ‘Corrected Flux’ eleanor light curves without further processing, highly variable stars that were observed in long-cadence only can trick the pipeline by mimicking an OED. Fig. 3 shows an example of this for the case of TIC 294179385, which is considered a FP by dave because of the nominal OED but was labelled as a planetary candidate (PC) after human inspection. Thus, the human vetter has to inspect the output of the Modelshift module and decides whether the detected features are genuine, also paying close attention to (i) the shape of the signal (whether it is U- or V-shaped); (ii) the depth of the primary signal (with respect to the stellar radius as provided by ExoFOP); (iii) the depth of the secondary signal (with respect to a typical expected depth of planet occultation, on the order of a few hundred parts-per-million); and (iv) the overall shape and amplitude of the light-curve variability both in- and out-of-transit.

• The variability is evaluated by the human vetter together with dave'’s Lomb-Scargle (LS) analysis (Lomb 1976; Scargle 1982) of the transit-masked light curve. This submodule provides a quantitative and qualitative criteria to evaluate the presence of possible light-curve modulations (LCMOD) due to intrinsic and/or rotational variability. If the detected modulations have the same (or half or double) period of the detected transit-like signal, they could be the result of gravitational (beaming effect and tidal ellipsoidal distortion) and/or atmospheric (reflected light and thermal emission) effects in a close binary star system (Morris & Naftilan 1993; Faigler & Mazeh 2011; Shporer 2017). This particular scenario is usually referred to as a BEaming, Ellipsoidal, Reflection Binary (BEER) binary. Whenever we found any suspicious modulations strictly related to the orbital detected period, we flagged the target using comments such as ellipsoidal variations (‘EV’) or synchronous variations (‘synch’). An example of a synchronous scenario is shown in Fig. 4.

Overall, while dave produces automated dispositions for each target, we mandate complementary human supervision for all targets due to the likelihood of systematics that can affect the pipeline’s classification. Importantly, our human vetters can override dave’s disposition and ultimately have the final word – any target in our catalog that exhibits potential signs of concern has been subjected to rigorous group discussions.

Aside from the vetting dispositions, for each TOI, we keep track of any noteworthy features using pre-defined acronyms and free-text comments as described in Table 1. As described below, we also updated the workflow presented in Paper I by introducing new diagnostic tests that are useful for the most challenging cases.

2.1 Ancillary information

In many cases, a target’s light curve or target pixel files are affected by prominent systematics and/or the detected transits have a low SNR compared to the baseline variability. This complicates the vetting procedure and can even make it unreliable altogether. To address this issue and confirm or dispel any concern, we use additional information beyond that provided by dave. For instance, the vetter manually checks whether the aperture mask used to extract the light curve includes nearby field stars that are bright enough to contaminate the inspected signal. Below, we briefly discuss new diagnostics that have been used in this work and we are currently implementing in dave to provide the vetters with a self-consistent tool without asking them to manually seek this ancillary information.

2.1.1 Unresolved sources

TESS has a large pixel scale, about 21 arcsec pixel⁻¹, with a focus-limited PSF. Hence, the flux measured in a single pixel might be contaminated by nearby background or foreground field sources. Based on our experience with dave and tess data, and depending on the particular target and sector, measuring a reliable photocentre offset of ∼5–10 arcsec (∼0.25–0.5 pixels) is relatively straightforward. In contrast, a bona-fide offset of ∼1–2 arcsec (∼0.05–0.1 pixels) is extremely challenging to measure. Thus, even if the photocentre module of dave does not measure a significant CO, there might still be sufficiently bright field sources that contaminate the target’s light curve and/or are too close to the target to reliably rule out as potential source of the detected transit-like signals.

In the former case, the additional light dilutes the transits, resulting in an underestimated planet radius (Ciardi et al. 2015). To account for this effect, we consult stellar catalogs (e.g. SIMBAD; Wenger et al. 2000, GAIA EDR3; Gaia Collaboration 2021) to check whether known sources fall within the immediate vicinity of the target. Based on the transit depth (δ) and magnitude difference between the target and resolved nearby field stars, the vetter would then investigate whether these alleged sources could have produced the observed transit signal. For a given target of magnitude m₀, we considered a threshold |$\Delta mag \, =\, -2.5\log _{10}\delta$|⁠. Thus only sources with a magnitude m_* such that |m_* − m₀| < Δmag could produce a signal with the same depth of the one observed. The scientific core team provides the vetters with the Δmag for each target. If some unresolved stars falls within the same pixel of the target, the vetter adds a comment Field Stars in Central Pixel (‘FSCP’). For completeness, the vetter will also flag a Field Stars in Other Pixel (‘FSOP’) whether a bright enough source falls within the aperture mask. This is done for the sake of completeness, but it is not a sufficient reason to rule out the target as an FP. This check is time-consuming and does not need the critical faculties provided by human inspection. In the future, we plan to provide the vetters with a simple tool that by performing a GAIA DR3 query, which returns all the stars within 5 pixels from the target. It will also mark those sources within the same TESS pixel (<21 arcsec) and colour each one according to their GAIA DR2 magnitude. The pipeline will then automatically flag any source inside and outside the target’s pixel that is bright enough to cause the observed dips in the light curve.

2.1.2 The background flux

TESS is in a stable, highly elliptical high-Earth orbit in a 2:1 resonance with the Moon. This orbital path ensures maximum sky coverage while minimizing the number of obstructions during data acquisition (Gangestad et al. 2013). However, this orbital path produces strong contamination in the TESS FFIs, mainly from zodiacal light and scattered light from Solar system objects (Sullivan et al. 2015). Hence, the background flux of TESS FFIs varies over the course of the ∼27-d observational window. To account for this, we inspect a 4-d-long section of the background flux centred on the time of the transit. This helps the vetter determine whether the transit signal seen in the light curve coincides with any background events. In fact, if there is a sudden change in the background flux at or near the time of the transit, it may introduce spurious signals into the light-curve mimicking or distorting the transit. Thus, for each detected transit, we check both the light curve and the background flux in the vicinity of the transit time. If unusual features and/or discontinuities appear in the background during a particular transit, the vetter will flag it as a potential issue. A clear example of an FP signal due to systematics in the flux background is shown in Fig. 5.

2.1.3 Pixel level light curve

Inspired by the LATTE³ pipeline developed within the Planet Hunters TESS project, we decided to include in our workflow a Pixel level light curve (PLL) analysis. The PLL plot shows the light curve for each individual pixel of the corresponding target pixel file. For further information, we refer the reader to Eisner (2022). We inspect the light curve for each pixel in the field of view and try to determine whether the transit occurs in the vicinity of the target or originates from another pixel that hosts another star – yet missed by dave’s photocentre analysis. This additional layer of scrutiny has proven to be very useful in cases where dave’s photocentre measurements were unreliable or difficult to interpret. For example, in some cases, the scatter in the individual photocentre measurements can be so large that it is practically impossible to distinguish between reliable and spurious measurements. This usually occurs when the individual difference images exhibit a complex pattern (or simply look like random noise) instead of a single bright spot superimposed on a uniform dark background. This is often due to low SNR transits caused by either (i) the presence of nearby field stars that are much brighter than the target itself (and/or are highly variable); or (ii) when the true source of the signal is next to a much brighter target star. In these cases, the PLL analysis helps determine whether some of the detected transits are affected by systematic effects and/or artefacts and ideally pinpoint the source of the signal.

Fig. 6 shows an example of such situation, highlighting how dave’s measured photocentres for TIC 256 886 630 are unreliable due to the poor quality of many of the individual difference images (scenario (ii) above). Here, the PLL analysis immediately reveals that the true (and faint) source of the observed signal is near the edge of the aperture mask – such that some of its signal does enter the aperture – whereas the (much brighter) target shows no transit-like signal. As a result, this target has been ruled out as an FP due to CO.

2.2 Dispositions and comments

According to the workflow described above, each of the 999 TOIs presented here was thoroughly examined by at least three vetters, including at least one member of the core science team. The purpose of this workflow is twofold: to distribute the total workload over a large group of people, saving significant time and to reduce the human bias that unavoidably affects inspection. Each vetter provides their evaluation (or disposition) of the TOI under scrutiny, according to the following prescriptions:

• if the TOI shows no anomalies at both the flux and the pixel level, then the signal is ranked as a PC. We also classify the target as PC by default if any of the following cases are met: (a) the light curve has low SNR resulting in a very shallow dip and there are no indications for a CO; (b) the photocentre analysis generates UC and the light curve does not show any obvious systematics; (c) the phase-folded sector-by-sector light curve shows no apparent transit signal (NT), and there are no known nearby sources bright enough to produce the transit depth. We note that an NT flag is not unexpected since dave analyses individual sectors instead of multisector data. As a result, low SNR and/or long-period candidates may not have sufficient per-sector SNR for dave’s tests.

• if the TOI does not pass the vetting procedure, then the signal is ranked as an FP. A significant CO represents one of the strongest clues for an FP scenario. A target is also classified as an FP when the phase-folded light curve exhibits a clear secondary eclipse (SS) or a significant OED. The latter is one of the most challenging features to distinguish as it is highly dependent on a quiet light curve;

• if dave generates a few red flags for a TOI, but there are no clear indications of a FP scenario, then the signal is ranked as a probable False Positive, pFP. For example, a pFP may arise when the TESS light curve has a low SNR and at the same time, we notice a potential secondary eclipse and/or the photocentre position seems to be slightly shifted towards a nearby field star. Long-period candidates are particularly difficult to analyse since the number of per-sector transits is small, and the measured photocentres might not be sufficient for a statistically significant evaluation. Often, there are only one or two photocentre measurements. In cases like these, we flag the candidate as a pFP instead of FP even if the photocentre analysis indicates an offset.

2.3 Automatic disposition generator

In addition to the analysis described above, we also followed an additional procedure, which we named automatic disposition generator (ADG), to automatically generate dispositions for TOIs based on the rankings of our vetters. For each TOI, we require dispositions from at least three vetters; the final disposition is determined by taking a weighted average of all vetters’ dispositions. A critical step is to provide the ADG with a reliability indicator for each vetter via a user score ϵ_i ∈ [0, 1] to account for varying levels of expertise within our team. As the volunteers who contributed to this work are the same as those who contributed to the Paper I catalog, we used the latter’s results to quantify the reliability of each vetter as follows.

For each vetter, we constructed their own confusion matrix, as shown in Table 2, using the final group dispositions of Paper I as our knowledge base.

In Paper I, the true PCs accounted for |$\sim 71~{{\ \rm per\ cent}}$| of the total catalog over a total of N_TOT = 999 targets. To account for the unbalanced nature of the knowledge base sample, we used the weighted average precision as the metric to assess each vetter’s level of reliability. Assume the i-th vetter ranked a certain number of targets in Paper I obtaining T_i(PC) number of correctly identified PCs, T_i(FP) number of correctly identified FPs, F_i(PC) number of incorrectly identified PCs, and F_i(FP) number of incorrectly classified FPs, then their score ϵ_i will be given by the following

where N_PC = 709 is the number of PCs in the catalog of Paper I, while |$\widetilde{N}_\text{FP}=290$| represents the number of both FPs and pFPs within the same catalog. Certainly, not all vetters have given the same number of dispositions, which may result in a non-uniform efficiency computation, but we ignore this as first-order approximation.

To calculate the weighted average of the overall disposition, we first convert labels into numbers, using the following convention:

Hence, we define the overall disposition as the vector |$\vec{D}$| determined by the average of given dispositions weighted over the fidelity of vetters,

where N_PC, N_pFP, and N_FP are the number of vetters who voted for PC, pFP, and FP scenario, respectively, while |$W\equiv \sum _{\ell =1}^{N}\epsilon _\ell$|⁠.

The final paper disposition is given using the following prescription:

In Table 3, we reported the scores of each superuser who contributed to this work.

ADG not only drastically reduces the time required to generate a uniformly vetted catalog, but it also allows for the reduction of human bias via a rigorous scientific approach. In this regard, ADG captures the ultimate essence of a Citizen Science Project.

3 PLANET PATROL

Cox et al. (2015) estimated that, on average and across all Zooniverse projects, citizen scientists inspected volumes of data equivalent to 34 yr of full-time work by a single expert. For example, volunteers have discovered 41 new long-period (Long-P) planet candidates in the Kepler data base (Wang et al. 2015) within the PHT project (Fischer et al. 2012). In 3 yr, citizen scientists involved in the PHT project helped the scientific team to discover hundreds of new planet candidates (Eisner et al. 2022a) along with a large number of eclipsing binary systems (Eisner et al. 2021), including a hierarchical triple-star system (Eisner et al. 2022b). Moreover, projects like the Visual Survey Group (Kristiansen et al. 2022) contributed to 69 peer-reviewed papers mainly focusing on exoplanets, multistellar systems, and unusual variable stars.

The Planet Patrol project was officially launched on the 2020 September 29 by the Zooniverse platform. The first stage of the project was aimed at improving the reliability of dave’s photocentre analysis by asking the trained users to evaluate the quality of the difference images generated by the centroid module. All users became acquainted with the workflow throughout brief vetting tutorials and F.A.Q. as well as numerous examples of FPs. More than 5600 volunteers examined ∼400 000 difference images in just 1 month, achieving 95 per cent of accuracy using as a knowledge base 198 classifications given by the science core team.

After removing the difference images flagged as poor by the volunteers from dave’s analysis, the photocentre uncertainty decreased by up to |$\sim 30~{{\ \rm per\ cent}}$| for the majority of the candidates (Kostov et al. 2022). After the completion of the first stage project (November 2020), many eager volunteers (superuser) expressed an interest in getting further involved in the vetting work. The superusers played a fundamental role in creating our first TT9 Catalog and repeated the feat by vetting the 999 TESS candidates and assisting the core science team in producing the catalog presented here.

3.1 Citizen scientists at work

The main key to success of a citizen scientist project is having constant interaction between the science core team and the superusers. Hence, we hold live weekly meetings where we discuss the progress of the project and provide superusers the opportunity to discuss any difficulties they may have encountered throughout their task. Because our team is made of people from around the world, one of the superusers, HADL, recorded all meetings and posted them on a dedicated YouTube channel. These recordings (currently private) are useful for people not able to attend the specific meeting and provide a valuable resource for newcomers.

Citizen Science has taught us that volunteers can not only offer invaluable assistance in the specific scientific task, but also, due to their diverse expertise, they could provide the scientific community important new ideas, resources and tools. For example, as the Google Sheets we used to keep track of our vetting process grew in content and complexity, it became difficult to find, create, analyse, and distribute the user dispositions and comments. To address this issue, one of us (RS) developed a custom vetting portal, Exogram (https://exogram.vercel.app), specifically designed to streamline the vetting process and facilitate group discussions. Exogram is hosted by Vercel, the data base and authentication is handled by Firebase, and parts of the backend logic is written in python.

Exogram’s homepage provides a user-friendly and intuitive interface that highlights targets that still need to be vetted by the user. It also allows the user to update their dispositions, search for TICs with specific dispositions or comments, and keep track of the overall vetting progress by all users. The website directly links each target to the dave-generated PDFs containing the vetting results and diagnostics stored on Google Drive. When creating dispositions, Exogram limits the user to three disposition options: FP, Planet Candidate (PC), and pFP. The user comments section accepts both a pre-defined list of machine-readable vetting acronyms (e.g. ‘CO’ for ‘Centroid Offset’) and free text.

In addition, Exogram allows the user to interactively inspect and manipulate the target’s light curve. The website downloads all available QLP data on demand from MAST, displays the corresponding normalized flux, centroid motion, and background flux for one or multiple TICs, and highlights the recorded momentum dumps. This allows the vetter an additional layer of scrutiny beyond that provided by dave, a complementary comparison between light curves produced by two different pipelines (eleanor versus QLP), and enables the user to explore and examine in detail the light curves of nearby targets. Fig. 7 represents one of the more interesting targets within our catalog for which the QLP’s light curve is completely different from that generated by eleanor.

4 THE CATALOG

The 999 candidates analysed in this work were drawn from the candidates provided by the ExoFOP TESS archive in the fall of 2020. They were selected by TIC number and do not overlap with our first TT9 catalog. Once each TOI had at least three dispositions, we ran ADG on the whole catalog. It generated 752 signals as PCs, 142 as FPs, and 105 as pFPs. Thus, overall approximately one in three planet candidates is an FP or a potential FP, a rate similar to that of Paper I. The most common comments within our catalog are ‘FSCP’ and ‘FSOP’, which occurred 628 and 481 times, respectively. This is expected, given that TESS targets are often contaminated by nearby background and/or foreground sources. We note that we only use these two flags as an extra layer of scrutiny – they are not sufficient to mark a candidate as an FP.

4.1 Planet candidates

Within our catalog, 752 TOIs passed all dave tests and human inspections as planet candidates. In this sample, there are 117 objects that have already been confirmed within the TESS scientific community or previously discovered by other exoplanetary surveys. 12 of the 752 PCs can be regarded as bona-fide, high-quality candidates, as they passed the dave test showing a clear box-shaped transit and high-significance on-target centroid measurements. In Table 4, we summarize the main properties of these 12 likely genuine planets. None of these 12 candidates has been confirmed by follow-up observations yet.

Apart from the ‘FSCP’ comments that are quite spread all over the catalog, the most common comments for our PCs are ‘LowSNR’ (270 times), ‘UC’ (247 times), ‘LCMOD’ (201 times), and ‘Vshape’ (147 times). The first two comments are strongly correlated because dave often generates UCs for signals with low SNR, thus making the classification challenging. As already discussed, in these cases, we automatically flag the target as PCs. The third most notable comment can either be caused by the inherent modulations of the targets under scrutiny or from the sources that contaminate the extracted light curve. Strong LCMODs can also completely hide shallow transits that could be identified after careful detrending. Finally, the flag for V-shaped transit is not a conclusive evidence to support a FP scenario. It only indicates that the two objects orbiting a common centre of mass have comparable sizes. Although this happens more frequently for a binary star system, we can not rule out a giant planet transiting its host star with a non-zero impact parameter (e.g. Smalley et al. 2011).

4.2 False positives

Our analysis classified 142 candidates as FP. Of these, we ruled out 118 targets as FPs due to a clear ‘CO’. While nearly 40 per cent of FPs in Paper I was flagged as ‘CO’, in this work, the rate has increased to approximately 83 per cent. The PLL analysis was likely essential for some targets that otherwise would have been flagged as PC because of poor centroid measurements. Furthermore, we believe that the observed percentage increase in the ‘CO’ flag is also due to the volunteers’ skill improvement after 2 yr of training.

The second most frequent FP indicator is the presence of a significant secondary eclipse (‘SS’, 33 targets), followed by OED (34 targets). Both of these flags are often accompanied with a ‘Vshape’ comment. All OED targets have been inspected for prominent modulations of the light curve.

We note that we vet all the TOIs presented here regardless of their current disposition on ExoFOP as done in Paper I. In particular, in Paper I, six confirmed planets were classified as FP due to a significant secondary eclipse at mid-transit. In this work, out of 142 targets, we labelled as FP TIC 427761355.01 and TIC 386259537.01 that have have confirmed as bona-fide planets by follow-up observations. The Modelshift of TIC 427761355.01 (or TOI-1518 b) shows a V-shaped transit with an SS exactly at half period. At this level of significance, we cannot distinguish a secondary eclipse from a planetary occultation, thus for consistency with our workflow we flag it as an ‘FP’. We also labelled WASP-169 b as an FP since its centroids module depicts a clear and reliable offset of the light photocentre at the time of transit. After inspecting this target with PLL, we discovered that there is a deeper transiting feature in the nearby pixel on the same period of WASP-169 b, which causes the overall centroid to shift.

4.3 Potential false positives

We labelled 105 TOIs in our catalog as pFPs. Our concerns and difficulties towards these targets are reflected in the most notable comment, ‘potential-CO’ (61 times). The prefix ‘potential’ qualitatively indicates that we are not fully convinced there is a significant photocentre offset due to ‘LowSNR’ (59 times) or prominent ‘LCMOD’ (40 times), which complicated the centroid measurements. It often happens that among many UC measurements, there are a handful that show a hint for a CO. In these cases, we could not eliminate our concerns with the PLL analysis either. However, it did help us identify as pFP three targets, which were previously classified as PC due to UC. The light curves of these targets usually do not show a clear transit (‘LowSNR’, 59 times), leading to 26 cases for which a potential secondary eclipse has been observed as well as 20 cases where an OED might be statistical significant.

4.4 Individual targets of interest

One of the most intriguing and worth noting planet candidate within our catalog is TIC 396720998.01, a sub-Jovian (⁠|$R_p\approx 0.35 \, R_J$|⁠) object orbiting a white dwarf (⁠|$R_* \approx 0.15 \, R_\odot , M_*\approx 0.5M_\odot$| and T_* ∼ 50 000K), according to the Tess Input Catalog. It has been observed by TESS in sectors 3, 4, 5, 30, 31, and 32. We also found additional transit-like features (≈6000 ppm) that may suggest a multiplanet system around this hot white dwarf as shown in Fig. 8. We flagged a V-shaped transit potentially due to the small size of the host star (⁠|$\approx 0.15 \, R_\odot$|⁠). This system could represent a perfect target to shed light on the evolution of a planetary system around Sun-like stars during the last stages of their evolution.

Among the TOIs listed in our catalog, we also kept track for planet candidates orbiting within the so-called habitable zone of their host stars. For each given star with known radius R_* and mass M_*, we calculated the inner and outer edges of its so-called habitable zone as defined by Kopparapu et al. (2013). As to the inner edge, we considered the runaway greenhouse at which the oceans evaporate entirely, while the outer edge was calculated considering the maximum greenhouse provided by a CO₂ atmosphere. We found two planet candidates that orbit the habitable zone of the stars TIC 271 971 130 and TIC 360156606.

TIC 271971130.01 is a planet candidate with R ≈ 1.6R_⊕ and P ≈ 19.3 d detected by the SPOC pipeline. TESS observed the target in sectors 1–13, 27, 29–37, and 39 at cadences of 2, 10, and 30 min. This TOI is marked in our catalog as a LowSNR candidate; in some sectors, it is quite challenging to see the transits. It is a faint star (TESS_mag = 13.5) for which we also flagged ‘FSCP’ and ‘FSOP’. Hence, the light curve is contaminated by nearby fainter sources (<15 TESS_mag) within the aperture mask and the same pixel. As discussed in Section 2, in cases like this, we consider the candidate as a PC by default. According to the TESS Input Catalog stellar parameters, the host star is a red M dwarf with |$T_* \approx 3187 \, K, R_*\approx 0.22 \, R_\odot$| and |$M_*\approx 0.20 \, M_\odot$|⁠. The candidate planet lies very close to the inner edge of the habitable zone of its star.

TIC 360156606.01 is a planet candidate with R ≈ 9R_⊕ and P ≈ 27.36397 d. Its host star has been observed by TESS in sectors 11 and 12 at a cadence of both 2 and 30 min, and in sector 38 at 20 s, 2- and 10-min cadence. The planet candidate is marked in our catalog as a LowSNR signal. Its light curve shows prominent modulations, which make the Modelshift analysis difficult. These modulations may originate from brighter sources that fall within the aperture mask. However, the detected transit is above the noise and quite clear. As discussed above, due to its long orbital period, the photocentre test from dave is inconclusive. According to the TESS Input Catalog Stellar Parameters, the host star is a red M dwarf with T_* ≈ 3055K, R_* ≈ 0.446R_⊙, and M_* ≈ 0.43M_⊙. This candidate has been recently confirmed by Mann et al. (2022) as TOI-1227 b within the TESS Follow-up Observing Program Working Group.

4.5 Comparison to dispositions based on machine learning

As we mentioned in the Introduction section, machine-learning-based pipelines are also effective in providing dispositions for a large sample of TOIs. To date, there are two main algorithms based on DL that have been explicitly tested on TESS data: the ASTRONET versions described in Yu et al. (2019) and in Tey et al. (2023); and Exominer (Valizadegan et al. 2022, see their Section 10). Yu et al. (2019) describe five different networks, with different tasks, ranging from the ‘triage’ model that only works on light curves and removes FP signal produced by instrumental artifacts, to vetting models that can also take into account analyses centroids’ positions and additional information. Their best vetting model achieves an average precision⁴ of 69.3 per cent and an accuracy of 97.8 per cent. Their algorithm has recently been improved by Tey et al. (2023), reaching a 99.6 per cent recall at a precision of 75.7 per cent. On the other hand, Exominer makes use of the unique elements of Kepler SOC/TESS SPOC data validation summary report in their original format. Exominer reaches an 88 per cent precision at the recall value of 73 per cent on TESS data. Following a similar approach to that of ASTRONET, Fiscale et al. (2021 and 2023, hereinafter F23), presented a deep-learning method to obtain dispositions from TESS data. By working only on the light curves, the model described in F23 achieves a precision of 87 per cent at recall value of 81 per cent. Note that applying Neural Network models described in the literature to an arbitrary data set is not straightforward: it requires additional work, even when the code is publicly available (as in the case of e.g. ASTRONET), including preforming the training from scratch (see e.g. the discussion in Visser, Bosma & Postma 2022). Besides reaching such good performances, the F23 model has the additional advantage of being developed within our research group and therefore can be immediately applied to the catalogs discussed in this work. We first tested the F23 network on the Paper I catalog in order to use the algorithm described in Section 2.3 to estimate its score, that is 0.73, hence similar or better than the one of a third of the superusers.

Therefore, we can compare the independent dispositions obtained by the neural network with the catalog obtained by exploiting citizen science, in order to check for consistency.

We show this comparison in the form of the confusion matrix in Fig. 9, where we consider our catalog’s outcome as ground truth. Hence, true positives (TP), true negatives (TN), FPs, and false negatives (FN) are computed with respect to our dispositions. Specifically, the TP and TN represent the fractions of TOIs classified by both our team and the network as PC and not PC, respectively. The FP indicates the fraction of TOIs we labelled as not planet candidate (i.e. we classified as FP or pFP) while the network predicts as PC, and FN indicates the number of TOIs that are indicated as PC in our catalog but are not identified by the network.

Furthermore, over half of the TOIs mislabelled as not planet by the network are flagged as LowSNR targets in this work, with half of these targets not showing any visible transit. In these cases, we decided to be conservative and pass the signal as a candidate if there are no other issue. The network, however, is trained on data sets where similar objects are not labelled as PC, hence it cannot provide the same disposition as us. Summarizing, we find that machine-learning approaches are promising, but they still need to be complemented with the study of the ancillary data available (such us the photocentre position) in order to provide final dispositions and validation, especially in the case of low SNR light curves.

5 DISCUSSION

In Fig. 10, we show the distribution of the 999 TOIs within the (P, R) plane. The figure highlights the high rate of FPs at short periods and large planetary radius. A potential explanation for this result may indicate that the majority of FP scenarios originate from close eclipsing binary systems. We also emphasize that our procedure automatically classifies all long-period candidates (>50 d) as PC because these objects have insufficient per-sector photocentre measurements and are usually flagged as’LowSNR’ candidate.

We also applied a two-sample Kolmogorov–Smirnov test to the orbital period distributions of PCs and pFPs-FPs obtaining a p-value less than 0.05. We repeated the test for the radii distribution between PCs and pFPs-FPs obtaining the same result. This suggests that the two samples come from different distributions within the fixed level of confidence. These trends are in agreement with those obtained in Paper I; in particular, we did not find any statistical deviations in the P and R distributions between the same classes of the two catalogs. This was expected since the methodology underlying both catalogs is practically the same. Hence, we merged the two catalogs in one sample containing 1998 uniformly vetted TESS candidates. We performed a statistical analysis of this sample by taking into account the statistical correlation between the orbital period and the planetary radius in the planet rate occurrence (Hsu et al. 2019). Hereafter, we will use (p)FPs when we refer to the both FPs and pFPs contained in the sample.

In Fig. 11, we show the difference between the occurrences of PCs and (p)FPs within the (P, R) diagram.

When considering the orbital period and the planetary radius at the same time, we observe that the (p)FPs still outnumber the PCs at short period (P ≲ 4 d) but the dependence on the planetary radius is more complex. In particular for P ≤ 4 d, the PCs underdense cells form a triangular shape region that overlaps the so-called Hot Neptune Desert. Demographic studies revealed a scarcity of discovered exoplanets within this region (Szabó & Kiss 2011; Beaugé & Nesvorný 2013). Hence, our analysis suggests that most of the planet candidate signals falling within the Hot Neptune Desert are consistent with FPs. This is also consistent with the results of Magliano et al. (2023) who classified a sample of Hot Neptune candidates using the same methodology. In particular, in their sample of TESS candidates with P ≤ 4 d and 0.27 ≤ R_p ≤ 0.44R_J, nearly 75 per cent of the investigated candidates were flagged as (p)FP. The rate occurrences obtained here could be also used as priors to develop a Bayesian pipeline aimed at vetting a batch of TESS candidates.

6 CONCLUSIONS

We presented our second catalog of 999 uniformly vetted transiting exoplanet candidates from TESS as part of the Planet Patrol citizen science project. We implemented new diagnostics within our workflow to help vetters scrutinizing the more challenging cases. We also introduced a more precise way of getting a final group classification based on vetter’s reliability. We marked 752 TOIs as planet candidates, of which 117 are confirmed planets. We also identified 12 planet candidates who passed all the vetting diagnostics placing themselves as high-priority targets to be confirmed. 142 TOIs have been classified as FPs mainly due to a clear offset in the measured photocentre and/or a significant secondary eclipse. To be consistent with our workflow, we found out that two targets labelled as FPs were true planets. Finally, 105 TOIs were flagged as potential FPs due to a potential CO or secondary eclipse dominated by LCMODs and/or systematics. Together with Paper I, this work creates a catalog of uniformly vetted TOIs that can be further used to prioritize targets amenable for follow-up observations. Additionally, the two catalogs can be utilized as a training set for machine-learning efforts aimed at full automation of the vetting process. This catalog is provided to the scientific community in the same format as the Table 5; the full table is available as supplementary material along with this manuscript. The files generated by dave are publicly available on the Exogram platform and will be also made available on ExoFOP-TESS as part of the metadata associated with each TOI.

SUPPORTING INFORMATION

TT9 2.0.csv

Please note: Oxford University Press is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.

Acknowledgement

This research has made use of the NASA Exoplanet Archive, which is operated by the California Institute of Technology, under contract with the National Aeronautics and Space Administration under the Exoplanet Exploration Program. We acknowledge the use of public TESS Alert data from the pipelines at the TESS Science Office and at the TESS Science Processing Operations Center (SPOC) and from the Massachusetts Institute of Technology Quick-Look Pipeline (QLP). This research has made use of the Exoplanet Follow-up Observation Program website, which is operated by the California Institute of Technology, under contract with the National Aeronautics and Space Administration under the Exoplanet Exploration Program. This publication uses data generated via the Zooniverse.org platform, development of which is funded by generous support, including a Global Impact Award from Google, and by a grant from the Alfred P. Sloan Foundation.

Software: dave (Kostov et al. 2019a), eleanor (Feinstein et al. 2019).

DATA AVAILABILITY

The data underlying this article will be shared on reasonable request to the corresponding author.

Footnotes

References