https://orcid.org/0000-0001-9518-9819
ABSTRACT
We describe the ongoing ‘survey for magnetars, intermittent pulsars, rotating radio transients, and fast radio bursts’ (SMIRF), performed using the newly refurbished UTMOST telescope. SMIRF repeatedly sweeps the southern Galactic plane performing real-time periodicity and single pulse searches, and is the first survey of its kind carried out with an interferometer. SMIRF is facilitated by a robotic scheduler which is capable of fully autonomous commensal operations. We report on the SMIRF observational parameters, the data analysis methods, the survey’s sensitivity to pulsars, techniques to mitigate radio frequency interference, and present some early survey results. UTMOST’s wide field of view permits a full sweep of the Galactic plane to be performed every fortnight, two orders of magnitude faster than previous surveys. In six months of operations from 2018 January to June, we have performed ∼10 sweeps of the Galactic plane with SMIRF. Notable blind redetections include the magnetar PSR J1622−4950, the RRAT PSR J0941−3942 and the eclipsing pulsar PSR J1748−2446A. We also report the discovery of a new pulsar, PSR J1659−54. Our follow-up of this pulsar at an average flux limit of ≤20 mJy, categorizes this as an intermittent pulsar with a high nulling fraction of <0.002.
1 INTRODUCTION
Since the serendipitous discovery of radio pulsars in the late 1960s, there have been about a dozen major surveys conducted with radio telescopes in pursuit of finding more (such as Large, Vaughan & Wielebinski 1968; Manchester et al. 1978, 1996; Edwards et al. 2001; Manchester et al. 2001; Cordes et al. 2006; Keith et al. 2010; see the pulsar survey data base1 by Lyon et al. 2016). While these surveys have primarily resulted in the characterization of a population of pulsars that have highly periodic emission, they have also led to the discovery of subclasses of pulsars whose radio emission is rather sporadic. Of the currently known ∼2500 radio pulsars (Manchester et al. 2005),2 only a minor fraction of them have seen to exhibit pulse sporadicity. This includes ∼200 nulling pulsars, ∼120 rotating radio transients (RRATs),3 5 intermittent pulsars, and 4 radio-loud magnetars (Backer 1970; Hesse & Wielebinski 1974; Ritchings 1976; Biggs 1992; Duncan & Thompson 1992; van Leeuwen et al. 2002; Camilo et al. 2006, 2007; Kramer et al. 2006; McLaughlin et al. 2006; Herfindal & Rankin 2007; Rankin & Wright 2007; Wang, Manchester & Johnston 2007; Rankin & Wright 2008; Herfindal & Rankin 2009; Burke-Spolaor & Bailes 2010; Levin et al. 2010; Lyne et al. 2010; Camilo et al. 2012; Gajjar, Joshi & Kramer 2012; Lorimer et al. 2012; Eatough et al. 2013; Olausen & Kaspi 2014; Kaspi & Beloborodov 2017; Lyne et al. 2017; Jankowski et al. 2019). It has long been speculated (McLaughlin et al. 2006; Keane & Kramer 2008; Lyne 2009; Lyne et al. 2017) that sporadic emitters could well outnumber regular pulsars in the Galaxy, as the selection biases in surveys hinder their detection.
Sporadicity in the observed pulse trains can arise for a number of reasons. It may be intrinsic to the pulsar, or it may be external, for instance, due to the presence of a binary companion or interstellar propagation effects. Intrinsic sporadic emission is seen in the case of nulling pulsars (Backer 1970; Wang et al. 2007), mode-changing pulsars, intermittent pulsars (Lyne 2009), RRATs (McLaughlin et al. 2006), and radio-loud magnetars (e.g. Camilo et al. 2012). The presence of a binary companion can occult the pulses in the case of eclipsing binary pulsars such as red backs and black widows (e.g. Bilous, Ransom & Demorest 2019; Main et al. 2018) and pulsar main-sequence binaries (e.g. Johnston et al. 1994; Shannon, Johnston & Manchester 2014). Accretion of matter from the companion can also temporarily halt radio emission, as seen in transitional millisecond pulsars (Archibald et al. 2009; Jaodand, Hessels & Archibald 2018). Strong lensing of pulses from the intrabinary material in black widow pulsars can also cause unusually bright pulses to be observed such as those seen in the case of PSR B1744−24A (Bilous et al. 2019). Finally, interstellar scintillation plays an important role for low dispersion measure (DM) pulsars by modulating the signal strength of the pulses and making them appear sporadic, on time-scales that depend on the receiver bandwidth. It is important to note that the observed pulsar intermittency is a strong function of the sensitivity of a telescope. For instance, a nulling pulsar observed with a telescope with lower sensitivity might just appear to be scintillating or mode changing with a more sensitive one. Nevertheless, the question remains of why such a varied population of sporadic emitters form such a minor fraction of the known pulsar population.
To understand the part played by selection biases, we review pulsar search surveys to date. We restrict ourselves to the searches of the southern Galactic plane (declination, δ < 0; Galactic latitude, |b| < 5) with the rationale that most of such sporadic emitters are non-recycled pulsars; the non-recycled pulsars are mostly concentrated in the Galactic plane; the southern sky contains the richest portion of the Galactic plane; and this portion of the plane has been searched less exhaustively than the northern counterpart. The surveys considered are reported in: Large & Vaughan 1971; Komesaroff et al. 1973; Davies, Lyne & Seiradakis 1977; Manchester et al. 1978; Dewey et al. 1985; Clifton & Lyne 1986; D’Amico et al. 1988; Johnston et al. 1992; Manchester et al. 1996, 2001; Keith et al. 2010; Keane et al. 2018.
These surveys resulted in discoveries of ∼1000 pulsars. Most of these surveys were performed with the Parkes telescope, which is a single dish instrument with a narrow field of view (FoV; 0.55 |$\rm deg^2$| for the 21-cm multibeam receiver, Staveley-Smith et al. 1996; |$\sim 1\,\rm deg^2$| for the 70-cm receiver). Although very sensitive, this resulted in slow survey speeds, taking on average ∼3 yr to complete, with an average time gap of 5 yr between successive surveys. Summing the observing times of all the surveys regardless of completeness, sensitivity, and operating frequency, an average total observing time spent on pulsar searches at a given point in the southern Galactic plane turns out to be only ∼2 h. This very low-cadence probe of the sky is the predominant reason why we are selectively biased against the detection of sporadic emitters.
Furthermore, none of the surveys (with the exception of SUPERB) employed real-time single pulse or periodicity searches. Consequently, pulsar discovery lagged observations, sometimes by of order years (e.g. processing of some surveys is ongoing, or being repeated, and pulsars are still being found in archival data, although this is partly a benefit of improvements in processing speed and search algorithms). This discovery lag significantly affects our capacity to perform confirmation observations of sporadic emitters such as magnetars, transitional millisecond pulsars, and long-term intermittent pulsars (e.g. Lyne et al. 2017), which are best followed up as soon as possible after the observations in which they were discovered. For instance, XTE J1810−197, the first magnetar found to emit pulsed radio emission, was in a radio-silent state for almost 11 yr until its recent revival in late 2018 (Desvignes et al. 2018; Gotthelf et al. 2018; Lower et al. 2018b; Lyne et al. 2018; Dai et al. 2019).
Performing rapid, regularly repeated, real-time surveys of the Galactic plane is an ideal way to probe the intermittent pulsar population. Selected interferometers offer this capacity, as they can combine large FoVs and high spatial resolution. The Australian Square Kilometer Array Pathfinder telescope (ASKAP) was used to perform one such survey recently (Qiu et al. 2019) in a fly’s-eye mode configuration where all antennas are used as independent single dish telescopes thereby offering a huge FoVs at the expense of reduction in sensitivity. Here, we present such a survey undertaken with the UTMOST telescope: SMIRF, the survey for magnetars, intermittent pulsar, RRATs, and fast radio bursts (FRBs). SMIRF is the first rapid, multipass, real-time Galactic plane survey with an interferometer. The survey is commensally operated with an FRB search programme (Farah et al. 2018) and a pulsar monitoring programme (Jankowski et al. 2019).
In the next section, we provide an overview of the SMIRF survey set-up and describe SMIRF’s scheduling and the data analysis pipeline. We then describe the survey sensitivity in Section 3 and report early results from the survey in Sections 4 and 5, before discussing future prospects for surveys of this type and drawing our conclusions.
2 SMIRF SCHEDULER AND SEARCH PIPELINES
2.1 SMIRF and the UTMOST backend
SMIRF is a multipass commensally operated Galactic plane survey with the capacity to search for both single pulse and periodic events in real time. The survey is conducted with the Molonglo Radio Synthesis Telescope (MOST) which is the east–west arm of the Mills Cross telescope, 40-km south-east of Canberra, Australia. The telescope is a transit instrument in the east–west direction and operates at a frequency of 843 MHz with a total observing bandwidth of ∼31.25 MHz, although only 50 per cent of the band is effectively useable due to radio frequency interference (RFI) and bandpass variations. However, the search is carried out over the entire bandwidth after applying dynamic RFI masks as needed. The telescope has a huge collecting area of ∼18 000 m2 split into 352 modules, each in turn containing 22 mechanically phased, right-circularly polarized ring antennas. The telescope has a wide FoV of ∼4° × 2° that enables fortnightly surveys over the southern Galactic plane bound by a Galactic longitude (l) and latitude (b): −115° ≤ l ≤ 40° and −4° ≤ b ≤ 4°, respectively. During the range of local sidereal times (LSTs) when the Galactic plane is not visible, a survey of the Magellanic clouds was also performed. The Galactic plane region is gridded into 512 pointings of 2° × 2° each (see Fig. 1 for the pointing grid) and an observing time of 6 min per pointing is chosen for computational ease and survey speed. This observing time is constrained predominantly by the transit time of sources across our primary beam, the random access memory (RAM) size in the computing nodes, the need for real-time performance of the search code, and the need for a rapid fortnightly survey. The LMC and SMC were gridded into 14 and 7 pointings, respectively, with an observing time of 12 min. For the operations from 2018 January to June, the total survey time has been 770 h, with 144 h on the Magellanic clouds and 626 h on the Galactic plane. A survey like SMIRF would not have been possible if not for the massively parallel computing cluster that forms the heart of the telescope’s backend. We provide a brief overview of the UTMOST backend here and redirect the reader to Bailes et al. (2017) for further details on the backend and the telescope hardware.
The grid of pointing centres for SMIRF, shown in equatorial coordinates. The 512 grid points are spaced on a 2° × 2° grid, although the primary beam of the telescope is larger (4° × 2°), so there is redundancy in the east–west direction. The extent in Galactic coordinates is −115° ≤ l ≤ 40° and −4° ≤ b ≤ 4°.
Signals from 352 antennas of the 1.6-km long east–west arm of the interferometer are digitized using ‘Lattice’ field programmable gate array boards, and fed to a polyphase filter bank (PFB) which channelizes the data from each antenna element to 320 channels with a bandwidth of ∼100 kHz each and a time resolution of 10.24 µs at 8-bit resolution. The data from the PFBs are then streamed into a computing cluster via a 10 Gigabit Ethernet network, initially to a set of 11 ‘Acquisition (AQ)’ nodes. The total incoming data rate is 22 GBps. These nodes perform phase and delay correction and also performs RFI mitigation for the data streams from each of the antennas. This is performed by a check for deviations from the Gaussian distribution of voltage by measuring their spectral kurtosis to search for sudden deviations in the total power in every frequency channel. Any such deviations are replaced by Gaussian random noise generated with the same mean and standard deviation as the incoming data stream.
The data are then streamed via a 56 Gbps Infiniband network to a set of eight ‘Beam Former (BF)’ computing nodes, after which data from all antennas for a specific frequency channel are collated into the same node for coherent beam forming. These nodes have a RAM of 128 GB and 4 nvidia geforce 1080-Ti GPUs each. The coarse frequency channels are appropriately phase rotated to form 352 single-channel ‘fan beams’, laid out across the inner 4° of the primary beam and equally spaced from the boresight (central) position. The single-channel fan beams are subject to a second corner turn where all the frequency channels for a subset of fan beams (∼45) are collated on to the same BF node. This coherent beamforming also acts as a second stage of RFI excision, as voltages from the RFI source destructively interfere when beamforming at the location of the pulsar. These beams are then searched in real time for transient events using a customized version of the graphics processing unit (GPU) based transient detection software, heimdall4 (Barsdell et al. 2012) and searched for pulsars using a new real-time pulsar detection software written for this survey called ‘SMIRFsoup’ (see Section 2.4), which is triggered automatically within a few seconds of the observation terminating. The eight BF computing nodes share the operating load of beamforming 352 fan beams and real-time searches for single pulses, facilitated by three GPUs. On average each node processes 44 fan beams. A fourth GPU on every node is used for real-time periodicity searches, along with an additional BF node that performs the periodicity searches of ‘bleeding sky positions’ (positions on the sky where data from multiple nodes need to be combined as detailed in Section 2.4). Note that while single pulse searches are performed during an ongoing observation, periodicity searches happen soon after the observation is over, in order to obtain all the data necessary for the FFT operation.
2.2 The SMIRF scheduler
A decision in mid-2017 to make MOST a transit-only instrument in the east–west direction (in order to improve its sensitivity by mitigating phasing issues and self-generated RFI) combined with a slow slew rate of the north–south (NS) drive (|$\sim 5^{\circ }\,\rm min^{-1}$|) meant that a sophisticated scheduler had to be developed that would maximize our observing cadence across different observing programmes. While an automated dynamic scheduler developed for robotic pulsar timing observations was already in use (see Jankowski et al. 2019),5 it had been developed for a fully steerable telescope and hence had to be extensively modified to work with SMIRF. A fully automatic scheduler was developed from scratch and added to the observing system that requires no human intervention.6 The scheduler automatically chooses appropriate sources/fields to observe across the observing programmes at UTMOST: namely the pulsar timing programme, the FRB search programme, and the SMIRF survey itself. The pipeline performs searches for single pulses from FRBs and pulsars in real time, periodicity searches for pulsars in real time, commensally times up to 4 pulsars inside the primary beam of the telescope, and sends email alerts of potential candidates and problems with the observing system, if any. The scheduler is capable of performing unsupervized 24 × 7 operations. Fig. 2 shows the cumulative completely automated operation of the telescope from 2018 January to June. Observations of the Galactic plane are the most prominent feature in the plot, represented by 512 ‘tiles’ or pointings (shown in green). Two similar high-density observing regions are due to the Magellanic clouds, which were added to the survey with an observing time of 12 min per pointing. Remaining regions are due to off-plane pulsars, follow-up of FRBs, and various calibration sources used for phasing the array. During normal operations when there are no hardware failures in the telescope drives, the scheduler is capable of monitoring ∼500 pulsars, ∼30 FRB fields, and ∼500 SMIRF pointings with an average cadence of 2 weeks or less, after accounting for average maintenance downtimes. A full description of the SMIRF scheduler is given in Appendix A.
An example of automated and unsupervized observations of pulsars, FRB fields, and SMIRF pointings made by the SMIRF scheduler between 2018 January and June. The algorithm for scheduler operations is provided in Section 2.2. The scheduler automatically monitors ∼500 pulsars, ∼20 FRB fields, and observes ∼500 SMIRF pointings with an average cadence of 2 weeks or less. Grid pointings have also been placed at the positions of the Large and Small Magellanic Clouds (green symbols, lower right).
2.3 The single pulse search pipeline
SMIRF uses the pipeline that is part of the UTMOST upgrade for the FRB search programme to also perform searches for single pulses from pulsars. The pipeline is similar to the one described in Caleb et al. (2016), with the addition of a real-time classification engine. (Farah et al. 2018). We provide a brief description of its operation as follows. As seen in Fig. 3, the beamformed data from 352 fan beams of an ongoing observation are streamed into a circular buffer from where they are presented to the heimdall single pulse search software. First, the software incoherently dedisperses the data for a number of DM trials for a range of DMs (0–2000 pc cm−3)7 with a tolerance of 1.25 (see section 2.3 of Levin 2012 and Morello et al. 2019 for details on the tolerance parameter). The dedispersed time series are then convolved with a series of increasingly wide top-hat (box-car) functions with the widths ranging from 327.68 µs to 42 ms, in steps of powers of 2, from which potential candidates are reported. Candidates with a signal-to-noise ratio greater than 9 are then provided to the ‘multibeam coincidencer’, which compares the DM and width of candidates detected at the same time in different fan beams to identify multibeam detections of individual events. A candidate simultaneously present in more than four beams is classified as RFI and is discarded from further analysis. The shortlisted candidates are then presented to a random forest machine-learning classifier (Breiman 2001) which classifies them as either astrophysical bursts (i.e. potential FRBs, pulses from potentially new pulsars, or pulses from known pulsars) or simply as RFI. The single pulse detection and classification take on average ∼15 s to complete. This provides sufficient time to instruct the AQ nodes to perform a dump of the corresponding baseband data from individual antennas, once a potential transient is detected with high detection significance. This has already been achieved for several of the UTMOST FRBs (Farah et al. 2018, 2019), and has resulted in very high time and frequency resolution studies of their properties. An email alert is sent for potential FRBs and new pulsars for further follow-up. A VOEvent trigger system (Petroff et al. 2017) for potential FRBs has been added recently (early 2019) for automated follow-up with partner telescopes.
A schematic of the SMIRF single pulse acquisition and data analysis pipeline. The primary beam is tiled with 352 fan beams, equally spaced across 4° of sky. The fan beams produce detected time series at 327 µsec and 97 kHz resolution, which are dedispersed in the DM range 0–2000 pc cm−3 and box car searched for single pulses down to S/N = 9. A random forest machine-learning algorithm classifies candidates as RFI and potential astrophysical signals. Pulses from known pulsars are flagged and retained separately. Candidate pulses may trigger a voltage dump if they have sufficiently high DM (as part of the search for FRBs; see Farah et al. 2018).
2.4 The SMIRF periodicity search pipeline.
To search for pulsars, we use a Fast Fourier Transform (FFT) based periodicity search software named ‘SMIRFsoup8’ which reuses the core modules of the GPU based pulsar search software, peasoup.9SMIRFsoup is a heterogeneous program that performs parallel multithreaded CPU–GPU operations to achieve real-time performance.
Figs 4(a) and (b) provide an overview of SMIRFsoup operations. Once an observation is complete, data of each fan beam resides on the hard discs of their respective BF nodes, as sigproc10 format filterbank files. The post-observation manager subthread of the scheduler (see Appendix A) then initiates an instance of SMIRFsoup on every BF node and reloads the data from all the fan beams (Nbeams) on to RAM of the CPU. The pipeline starts by performing an incoherent dedispersion of the data for a number of DM trials (Ntrials) over the range of (0–2000 pc cm−3) with a DM tolerance of 1.25. Once dedispersed, the data are summed in frequency to obtain (Nbeams × Ntrials) time series.
(a) A schematic of the SMIRF real-time fold-mode data analysis pipeline. As with the single pulse pipeline (see Fig. 3), the data of each fan beam is first dedispersed for NDM trials to obtain NDM time series. These are stitched together over a grid of tracks of points on the sky (i.e. in RA and Dec.) and sent to the GPU kernel for processing. (b) Continuation of Fig. 4(a). Once the time series formed by stitching filterbanks are loaded on to the GPU, the data are Fourier transformed and the spectrum is subjected to a harmonic summing kernel that sums up to 16 harmonics. A peak detection algorithm shortlists high signal-to-noise ratio candidates from the harmonic summed spectrum. These candidates are ‘coincidenced’ with other DM trials and stitches to produce a final candidate list that is given for observer’s discretion.
An initial RFI rejection algorithm performs an FFT of the zero-DM time series of all the available fan beams, and identifies periodic signals with an FFT S/N > 9. A local coincidencer then gathers the zero-DM ‘candidates’ from every fan beam. It shortlists the candidates that are found in more than six fan beams, since, for the survey observing time per pointing, it is not possible for a pulsar to be present in more than six fan beams (see below). These shortlisted candidates are then tagged as periodic RFI and are used to create a dynamic RFI mask for the pulsar search. The identified RFI periods are also stored into an RFI data base. The RFI data base thus keeps a track of all the common RFI periods that have been identified by several observations.
For telescopes with a large FoV such as UTMOST, tracking a boresight sky position of any given pointing causes a differential motion of the rest of the sky with respect to the boresight. As the FoV of UTMOST is tiled with narrow, highly elliptical fan beams, off-boresight sky positions generally drift from one fan beam to another during an observation. A potential pulsar hence might traverse several fan beams in an observation, which then need to be ‘stitched’ together along the path of traversal before one can use conventional FFT techniques for pulsar searches. In order to perform our periodicity searches, we grid the visible sky into regions of ‘similar’ traversals (Nregions) and model the fraction of time spent by each such region inside a given fan beam. We compute the trajectory of each grid region for an observation and perform stitches along them to produce stitched beams (Nstitches). We use a telescope-specific coordinate system of NS tilt and east–west tilt (Meridian distance; MD) to stitch fan beams (see Bailes et al. 2017 for a brief overview of the telescope specific coordinates).
The stitching algorithm is a core feature of SMIRF. Typically it operates with NFB = 352 fan beams tiled across the 4° wide primary beam in the east–west direction. The algorithm first obtains the extent (Ei) of each fan beam (|$F^{\rm init}_{\rm i}$|) in hour angle (HA) and declination (Dec.). This Ei is then gridded into a number of ‘regions’ (Ri) in declination, with a grid spacing of 23 arcsec (i.e. half the east–west fan-beam spatial resolution). For every time section (Ti = 6 s; chosen on the basis of how quickly off-boresight regions move, and the computation time), the instantaneous HA of each Ri is computed, and the fan beam Fj to which this HA corresponds is saved. This loop yields the traversal times of each NFB × Ri. A secondary loop takes such trajectories and groups them into region groups (Gk) which follow sufficiently similar trajectories, in order to minimize the number of FFT operations. At the end of this procedure, we obtain a list of Gk, each with their computed mean traversals for the observation such that no region (Ri ∈ Gk) suffers an incorrect fan-beam stitch for more than 10 per cent of the observing time. This limit is a trade off between real-time operation and the S/N degradation of a potential pulsar.
The stitched beams corresponding to each Gk are then searched independently for pulsars. Since only a subset of beams are processed by every BF node, Gk that reside in boundary fan beams tend to ‘bleed’ into other nodes. The fan beams corresponding to such Gk are transferred via the Infiniband network to an additional BF node, where similar pulsar searches are performed. Thus there are a total of nine BF nodes that share the workload of pulsar searching. Our heterogeneous CPU–GPU architecture allows for parallel stitching and pulsar searching operations. For every Gk, the stitcher constructs the appropriately dedispersed NDM × NFB time series to produce NDM stitched time series data that are then transferred to the GPU.
The pulsar search is independently performed for every NDM time series. It starts with a de-reddening routine that uses a median-of-means algorithm to estimate the red noise power in each DM trial. This is followed by a 221-point real-to-complex forward FFT. A square law detector then estimates the power spectrum and feeds it to a harmonic summing kernel. The kernel sums up to 16 harmonics (in powers of 2), subsequent to which peaks with S/N ≥ 9 are identified as potential pulsar candidates. A harmonic distiller compares the estimated periods of the candidates and removes any candidates that are harmonics of another candidate with the fundamental period. The candidates are then passed to the multistage ‘coincidencer’. While the GPU performs the operation of ‘pulsar searching’, the stitches necessary for the next Gk are performed on the CPU in parallel and made available before the current search ends. Hence, there is no time lost waiting for the stitching operation. All candidates go through several stages of coincidencing before any is considered a potential pulsar. Once candidates for every DM trial are obtained, a DM-coincidencer compares the S/Ns of the candidates with the same period in every DM trial. The DM with the highest S/N for a given period is chosen and the other candidates are discarded. The DM-coincidenced candidates (Ncands) are stored on the CPU RAM.
Once all the stitches for a given observation on a given BF node have been processed, that node streams its candidates to all other nodes for a global multibeam coincidence operation. The candidate periods are first cross-checked with the RFI data base for any known interference which may have seeped through the initial dynamic-RFI rejection, and are removed. Secondly, a sky-position-coincidencer groups candidates that possess the same DM and period from multiple stitches. The sky position of the one with the highest S/N is compared with the rest. Since it is a priori known that the maximum number of fan beams that a potential pulsar can move through in a given SMIRF observation is six (based on the sidereal rate), any candidate found in stitches whose |$F^{\rm init}_{\rm i}$| are more than six fan beams away are discarded as RFI.
Candidates that pass through all the shortlists are then saved to a MySQL data base (see Appendix A) as potential pulsars, along with their corresponding stitching information and other metadata. This triggers the post-observation manager to run a ‘Folding’ thread, which reads the potential candidates and the raw filterbank files, and stitches the appropriate fan beams (this time with full frequency information). The manager then loads the stitched beam into the pulsar folding software, dspsr11(van Straten & Bailes 2011), which then saves folded candidate archives at the candidate DM, period, and sky position. These candidate archives and the raw filterbanks are then collated on to the SMIRF management node, where a daemon produces candidate plots using the pdmp program of the psrchive software package (Hotan, van Straten & Manchester 2004).12 The observer can later use a custom candidate viewer to view the candidate plots across all SMIRF surveys, along with other metadata such as the stitching information, names of known pulsars in the beam, and the tied-beam S/N of pulsars that were commensally timed. When viewing any candidate, a ‘pulsar identifier’ compares the candidate parameters to the psrcat data base (Manchester et al. 2005) and flags that it is likely a known pulsar. In a typical observation, the FFT search returns a few hundred candidates, most of which are coincidenced out by the multibeam coincidencer. The typical number of candidates out of the coincidencer is of order 20, which are then folded. The number of unknown pulsar candidates that have a folded signal-to-noise ratio >9 is typically less than 10.
On average, the periodic searches happen at 0.8× real-time, the folding of candidates under usual RFI conditions happen at 0.1× real-time. Hence, the periodicity search pipeline for an observation is completed by the time the data recording for the next pointing is concluded. The disc size in every node is big enough for as worse as 3× real-time operations, to ensure unsupervized operations even in the case of rampant RFI causing the pipeline to run slower than usual. In case that the observations are about to be terminated due to impending disc storage overflow, an email is automatically sent for manual intervention.
2.5 Data storage
After folding potential candidates in high time and frequency resolution, the raw filterbank files are decimated to a time resolution of 655.36 |$\mu \rm s$| and a frequency resolution of 0.78125 MHz at 8-bits per sample. These filterbanks are archived to LTO-6 magnetic tapes for long-term storage and reprocessing. If we find a potential pulsar in a future SMIRF survey, the data storage allows for folding all the previous surveys at the topocentric period of the candidate, hence obtaining a confirmation of the candidate and/or obtaining an initial estimate of its nulling fraction. The decimated filterbanks for all the beams in a 6-min pointing sums upto a size of 7.7 GB. This on average amounts ∼4 TB per SMIRF survey. These data are stored in 2 × 2.5 TB LTO-6 tapes.
3 SURVEY SENSITIVITY
Although the system can record up to 31.25 MHz of bandwidth, the high degree of gain variations across the bandpass effectively reduces the usable bandwidth to only about 16 MHz. The system equivalent flux density (SEFD =G/Tsys) at UTMOST is variable in time due to issues including incorrect phasing, phase degradation since the last calibration, ring antenna misalignment, self-induced RFI and cross-talk, high degree of variations in receiver box performance, and mechanical factors such as deformations in the telescope structure (see Caleb et al. 2016 and Bailes et al. 2017 for more information). This makes it impossible to provide a single canonical sensitivity limit for the survey. Since the degradations from these contributors is difficult to measure realistically with the required cadence, an approximate SEFD value is thus obtained by using regular observations of a high DM pulsar, PSR J1644−4559, and using the radiometer equation to obtain the SEFD of the telescope, assuming a flux density of Smean = 960 mJy for the pulsar at 843 MHz (from Jankowski et al. 2019; see Fig. 5).
System equivalent flux density (SEFD) of UTMOST derived from observations of the bright southern pulsar J1644−4559. The dark blue points mark the measured sensitivities and the dark blue line shows the mean SEFD of 170 Jy. The light blue band shows the 1σ deviation of the SEFD estimates. The RFI environment at UTMOST, which includes traffic from mobile handsets in the 820–850 MHz band, causes considerable day-to-day variation in the system sensitivity.
Assuming an effective bandwidth Δν = 16 MHz and a system equivalent flux density of 170 Jy, we expect to blindly redetect ∼140 known pulsars at the present sensitivity, without correcting for position of the pulsar in the primary beam. Fig. 6 provides a plot of our sensitivity to pulsars as a function of pulse period for various values of DM for a fixed duty cycle of 5 per cent. The flux densities of known pulsars that are currently part of the timing programme, scaled to the position in the primary beam of the pointing, is also shown. Table 1 provides a comparison of the survey parameters to the HTRU survey.
The mean limiting flux density for the UTMOST SMIRF survey shown as a function of pulsar period for various values of DM. Values shown here assume a pulse duty cycle of 5 per cent. The bands of the sensitivity curves are due to the uncertainty in the telescope’s SEFD as shown in Fig. 5. The crosses indicate pulsars that are part of the UTMOST timing programme and the colours indicate their corresponding DM curves.
Comparison of SMIRF survey parameters to the HTRU survey (Keith et al. 2010), which is the most recent all-sky pulsar survey with the Parkes telescope.
| Parameters | SMIRF | HILAT | MEDLAT | LOWLAT |
|---|---|---|---|---|
| Instrument | UTMOST | Parkes | Parkes | Parkes |
| Central frequency (MHz) | ∼835 | ∼1352 | ∼1352 | ∼1352 |
| Bandwidth (MHz) | ∼16 | 340 | 340 | 340 |
| Δchan (kHz) | 100 | 390.625 | 390.625 | 390.625 |
| Survey region | −115° ≤ l ≤ 40° | δ < 10° | −120° ≤ l ≤ 30° | −80° ≤ l ≤ 30° |
| |b| ≤ 4 | |b| ≤ 15 | |b| ≤ 3.5 | ||
| number of bits | 8 | 2 | 2 | 2 |
| τobs (s) | 360 | 270 | 540 | 4300 |
| τsamp (µs) | 327.68 | 64 | 64 | 64 |
| Sensitivity limita (mJy) | 15 ± 5 | ∼0.18 | ∼0.25 | ∼0.35 |
| Parameters | SMIRF | HILAT | MEDLAT | LOWLAT |
|---|---|---|---|---|
| Instrument | UTMOST | Parkes | Parkes | Parkes |
| Central frequency (MHz) | ∼835 | ∼1352 | ∼1352 | ∼1352 |
| Bandwidth (MHz) | ∼16 | 340 | 340 | 340 |
| Δchan (kHz) | 100 | 390.625 | 390.625 | 390.625 |
| Survey region | −115° ≤ l ≤ 40° | δ < 10° | −120° ≤ l ≤ 30° | −80° ≤ l ≤ 30° |
| |b| ≤ 4 | |b| ≤ 15 | |b| ≤ 3.5 | ||
| number of bits | 8 | 2 | 2 | 2 |
| τobs (s) | 360 | 270 | 540 | 4300 |
| τsamp (µs) | 327.68 | 64 | 64 | 64 |
| Sensitivity limita (mJy) | 15 ± 5 | ∼0.18 | ∼0.25 | ∼0.35 |
aAssuming DM =100 pc cm−3, a period of 1 s and a 5 per cent pulse duty cycle.
Comparison of SMIRF survey parameters to the HTRU survey (Keith et al. 2010), which is the most recent all-sky pulsar survey with the Parkes telescope.
| Parameters | SMIRF | HILAT | MEDLAT | LOWLAT |
|---|---|---|---|---|
| Instrument | UTMOST | Parkes | Parkes | Parkes |
| Central frequency (MHz) | ∼835 | ∼1352 | ∼1352 | ∼1352 |
| Bandwidth (MHz) | ∼16 | 340 | 340 | 340 |
| Δchan (kHz) | 100 | 390.625 | 390.625 | 390.625 |
| Survey region | −115° ≤ l ≤ 40° | δ < 10° | −120° ≤ l ≤ 30° | −80° ≤ l ≤ 30° |
| |b| ≤ 4 | |b| ≤ 15 | |b| ≤ 3.5 | ||
| number of bits | 8 | 2 | 2 | 2 |
| τobs (s) | 360 | 270 | 540 | 4300 |
| τsamp (µs) | 327.68 | 64 | 64 | 64 |
| Sensitivity limita (mJy) | 15 ± 5 | ∼0.18 | ∼0.25 | ∼0.35 |
| Parameters | SMIRF | HILAT | MEDLAT | LOWLAT |
|---|---|---|---|---|
| Instrument | UTMOST | Parkes | Parkes | Parkes |
| Central frequency (MHz) | ∼835 | ∼1352 | ∼1352 | ∼1352 |
| Bandwidth (MHz) | ∼16 | 340 | 340 | 340 |
| Δchan (kHz) | 100 | 390.625 | 390.625 | 390.625 |
| Survey region | −115° ≤ l ≤ 40° | δ < 10° | −120° ≤ l ≤ 30° | −80° ≤ l ≤ 30° |
| |b| ≤ 4 | |b| ≤ 15 | |b| ≤ 3.5 | ||
| number of bits | 8 | 2 | 2 | 2 |
| τobs (s) | 360 | 270 | 540 | 4300 |
| τsamp (µs) | 327.68 | 64 | 64 | 64 |
| Sensitivity limita (mJy) | 15 ± 5 | ∼0.18 | ∼0.25 | ∼0.35 |
aAssuming DM =100 pc cm−3, a period of 1 s and a 5 per cent pulse duty cycle.
4 SMIRF SURVEYS TO DATE
A pilot survey (SMIRF-0) was conducted to understand the real-time performance and feasibility of the survey in mid 2017, while the telescope was still capable of slewing in east–west by differential rotation of its ring antennas. Although this survey detected only ∼30 pulsars owing to the low sensitivity of the telescope at the time (SEFD ∼ 400 Jy), it was a technological milestone, as it proved we could perform a survey of the Galactic plane (gridded as Fig. 1) in ∼10 d while searching for pulsars and FRBs in real time.
Soon after the telescope was made a transit-only instrument in the east–west direction, our sensitivity improved by a factor of 2 to SEFD = 170 ± 30 Jy. After the development and testing of the fully automated scheduler, the SMIRF survey has been undertaken almost daily since 2018 January, apart from downtime due to telescope maintenance and/or repair. At present, the scheduler regularly times ∼500 pulsars with a cadence of two weeks or better. This has resulted in the detection of a recent six more glitch discoveries (Lower et al. 2018a, Lower et al., submitted) to add to the eight already reported (Jankowski et al. 2019) proving the capabilities for such a massive scale pulsar monitoring programme. A new FRB was also discovered (FRB20180528; Atel: #11675) commensally during a ‘wait time’ FRB search observation that was automatically scheduled while waiting for PSR J0702−4956 to enter the beam. The FRB was discovered with the pipeline described in Section 2.3 (see Farah et al., in preparation for complete details).
In the 6 months of operations from 2018 January to June, we have performed ∼10 surveys of the Galactic plane with SMIRF. An important caveat is that, as one can see from Fig. 7, the survey completeness is not uniform. There are a number of reasons for this. First, UTMOST is a transit telescope, and the survey time is constrained by the variable sky drift rate as a function of δ. This causes parts of the Galactic plane with δ > −30° to be less frequently probed. Secondly, the desire/strategy to observe some of the pulsars at crucial local sidereal times at high cadence, reduces the number of surveys that can be performed around the RA of those pulsars. Some pulsars, such as the magnetar PSR J1622−4950, and the flux calibrator J1644−4559 are observed across the FWHM of the telescope owing to the multiple science goals, which also reduces survey time near the RA of those sources.
The relative fraction of survey pointings shown as a function of the number of times each has been observed. Over 98 per cent of the SMIRF pointings have been observed more than 10 times during 10 sweeps of the Galactic plane in the period 2018 January to June.
We provide the initial results of the redetection of known pulsars and a new intermittent pulsar in the next sections.
5 INITIAL RESULTS
The survey has resulted in the redetections of 51 pulsars in single pulse mode and at least 137 pulsars in fold mode, which is ≈98 per cent of the pulsars we expected to find, based on the system sensitivity, the integration times per pointing, and the position of the pulsar in the beam.
Three of these were particularly interesting: first, the redetection of an RRAT, PSR J0941−3942, during a ‘wait time’ single pulse search observation (Fig. 8), secondly, the reduction of the magnetar PSR J1622−4950, during its revival as a radio pulsar in late 2017 (Fig. 9), and finally, a redetection of PSR J1748−2446A, an eclipsing millisecond pulsar in the Terzan 5 globular cluster (Fig. 10), via giant pulses emitted during egress.
An example of a blind detection of an RRAT: a single pulse detection of the RRAT PSR J0941−3942.
Example of a blind detection of known pulsars as UTMOST candidates. The left-hand panel shows a phase versus frequency plot of the redetection of a bright pulsar, PSR J0837−4135. The right-hand panel shows a phase versus frequency plot of the redetection of a magnetar, PSR J1622−4950, when it was caught in a radio-loud state.
Phase versus frequency and time plots along with the pulse profile of the redetection of an eclipsing binary millisecond pulsar, PSR J1748−2446A which resides in the Terzan 5 globular cluster (the acceleration due to the binary orbit can be seen as a bend in the pulse phase versus time panel of the plot). This pulsar is normally invisible to Molonglo’s sensitivity, however, detected now from its giant pulses during eclipse egress.
5.1 A new intermittent pulsar, PSR J1659−54
As discussed earlier, the SMIRF scheduler regularly times ∼500 pulsars in addition to performing 2-week Galactic plane surveys. In such pointings, real-time single pulse searches for pulsars and FRBs happen simultaneously. In one such 6-min observation of PSR J1711−5350, we obtained real-time detection of single pulses from a source that was drifting across the fan beams at the sidereal rate. Since this pointing was outside the SMIRF pointing grid, real-time pulsar searches were not performed. However, we performed an offline pulsar search and discovered a new pulsar, PSR J1711−54.
The pulsar was seen to be nulling during the detection observation. Out of the 6 min observing time of the field, the pulsar was seen to be ‘ON’ for only ∼20 s, out of which we obtained eight single pulses whose S/N was greater than our detection threshold (see Fig. 11 for examples of single pulses and Fig. 12 for the folded profile). The best-fitting pulsar parameters are provided in Table 2.
Examples of single pulses detected from the new intermittent pulsar, PSR J1659−54. The top section of both the panel shows the single pulse integrated profile while the bottom section shows the (dispersed) pulse as a function of time.
Detection plot of PSR J1659−54 folded with single pulse time integrations. The top left panel shows the signal-to-noise ratio as a function of deviation from best DM and period. The middle panel shows the profile of the pulsar as a function of time and frequency. The bottom panel shows the integrated pulse profile of the pulsar.
Parameters for PSR J1659−54.
| Best-fitting parameters | |
|---|---|
| Pulsar name ... | J1659−54 |
| Pulse frequency, ν (s−1) ... | 1.8363(5) |
| Right ascension, α (hh:mm:ss) ... | 16:59:23(5) |
| Declination, δ (dd:mm:ss) ... | −54(2) |
| Dispersion measure, DM (pc cm−3) ... | 134(15) |
| Epoch (MJD) ... | 58196.8 |
| Best-fitting parameters | |
|---|---|
| Pulsar name ... | J1659−54 |
| Pulse frequency, ν (s−1) ... | 1.8363(5) |
| Right ascension, α (hh:mm:ss) ... | 16:59:23(5) |
| Declination, δ (dd:mm:ss) ... | −54(2) |
| Dispersion measure, DM (pc cm−3) ... | 134(15) |
| Epoch (MJD) ... | 58196.8 |
Parameters for PSR J1659−54.
| Best-fitting parameters | |
|---|---|
| Pulsar name ... | J1659−54 |
| Pulse frequency, ν (s−1) ... | 1.8363(5) |
| Right ascension, α (hh:mm:ss) ... | 16:59:23(5) |
| Declination, δ (dd:mm:ss) ... | −54(2) |
| Dispersion measure, DM (pc cm−3) ... | 134(15) |
| Epoch (MJD) ... | 58196.8 |
| Best-fitting parameters | |
|---|---|
| Pulsar name ... | J1659−54 |
| Pulse frequency, ν (s−1) ... | 1.8363(5) |
| Right ascension, α (hh:mm:ss) ... | 16:59:23(5) |
| Declination, δ (dd:mm:ss) ... | −54(2) |
| Dispersion measure, DM (pc cm−3) ... | 134(15) |
| Epoch (MJD) ... | 58196.8 |
Follow-up observations were conducted soon after the discovery with UTMOST with real-time single pulse and fold-mode searches. 19 observations of 6 min each, were performed, all of which have resulted in the non-detection of the pulsar to a single pulse and fold-mode S/N threshold of 9. This places an initial estimate of the nulling fraction of this pulsar to be <0.002 at an average flux limit of ∼15 mJy. The pulsar is yet to be redetected but regular follow-up observations are being scheduled with the UTMOST telescope by the SMIRF scheduler. Even without confirmation observations, we are confident that this candidate is a genuine pulsar because of its non-zero DM, broad-band emission, its movement from one fan beam to another at the sidereal rate, and not close in frequency to any RFI detected with other beams or in the RFI data base.
6 FUTURE PROSPECTS
Although SMIRF is already performing well, there are a number of improvements that can be done to improve its pulsar discovery efficiency. In this section, we describe improvements to the data analysis pipeline that are being implemented or will be implemented in the near future.
The present pipeline performs periodicity searches over a constant DM range of 0–2000 |$\rm pc \,cm^{-3}$| with a tolerance of 1.25. For some parts of the Galactic plane, searches to such high DM are inefficient, as electron density models for the ISM suggest a maximum DM in a given direction which is much smaller than this. Limiting the maximum DM to be searched for a given pointing, might be a better strategy to save computational resources that could otherwise be used to perform a search with, for instance, a better DM tolerance or to probe a modest acceleration range. For pointings toward e.g. the Magellanic clouds or globular clusters that are at known distances, it is also beneficial to perform coherent dedispersion of the filterbanks at the central DM of the galaxy or cluster, respectively, thereby decreasing the dispersion smear of the pulsar.
The presence of bright pulsars in the FoV (such as PSR J0835−4510 and PSR J1644−4559), can overwhelm the detection pipeline, by producing orders of magnitude more candidates than it can presently efficiently coincidence. Bright pulsars can appear as candidates in multiple stitches, up to several arcminutes away from the position of the pulsar. The coincidencer necessarily then treats each candidate as a potential pulsar, leading to several hundred candidates needing to be manually scrutinized. One can solve this problem by identifying the fan beams these bright pulsars traverse through, and ignore any stitch that passes through these fan beams for pulsar searching. While such a strategy would affect detections of new pulsars near such bright ones, the number of candidates to look at would be greatly reduced.
With the present pipeline, it is left to the observer to perform the manual task of searching through multiple epochs of a particular pointing to find out if an interesting candidate was also detected in early observations. Automating this task has the potential to identify subthreshold candidates that were detected from previous surveys, but were ignored for manual scrutiny owing to low signal-to-noise ratios. These candidates would then act as verification of an interesting candidate, thereby potentially helping confirm a new detection.
The present pipeline only performs a search for pulsars in the Fourier domain via FFT techniques. As is well known, this is not ideal for detecting long period pulsars as the associated part of the spectrum is dominated by red noise. The technique can also struggle to detect pulsars with very narrow duty cycles due to the finite number of spectral harmonics that are incoherently summed to search for a peak. Fast folding searches offers the possibility to overcome such loss in sensitivity, albeit more computationally intensive. Implementing a parallel FFA algorithm will help us remove the above mentioned bias in our processing pipeline.
While the above mentioned improvements are for the data analysis and candidate identification pipeline, SMIRF is currently mainly limited by the low sensitivity of the survey, which can only be addressed by improving the telescope hardware. While the searches with the east–west arm are to be continued in the near future, the NS arm of the telescope is currently being refurbished to transform the telescope into a 2D interferometer (UTMOST-2D), aimed at precise localization of real-time detected FRBs. The NS arm is 1.55 km long and 11.7 m wide. The basic element of the NS arm (a ‘cassette’) is a 1.4 m long section of the NS arm, fitted with 8 dual-polarized clover-leaf antennas that are beam formed in analogue to provide a primary beam that covers approximately ∼13° × 2°. The design has a system temperature of ≈70 K and a bandwidth of 45 MHz centred at 830 MHz. The first stage of the project that is ongoing (Day et al., in preparation), is aimed at installing 70 cassettes along the arm by end of 2019. Once completed, such a system will cover ∼100 m of the NS arm but will already possess a sensitivity comparable to the existing East-West arm.
It is interesting to speculate how much better SMIRF surveys would fare if performed with the NS arm. SMIRF in the EW arm is highly computationally expensive as pulsars traverse from one fan beam to another within the duration of a transit observation. This requires regridding the FoV into several regions, computing their traversals, and independently performing periodicity searches along each traversal (stitches; see Section 2.4) On average, of order Nstitches ≈ 5000 stitches need to be performed for every observation, and hence ≈5000 Fourier transform searches are performed. This complexity will be greatly reduced with the NS arm as the fan beams with the NS arm are oriented along the EW direction, and hence points on the sky, in most cases, would stay within the same fan beam for the entire observation. This reduces the number of FFT operations from Nstitches to just NFB which for ≈1400 EW fan beams, yields a factor of ≈4 reduction in the computational requirements to perform the same survey. The released computational resources could then perform e.g. parallel FFA searches or acceleration searches.
If the entire 1.55 km long NS arm was tiled with cassettes, that would present an increase in sensitivity of about a factor of ∼10 compared to the present EW arm, and one could perform a survey to a sensitivity limit of 0.25 mJy. This sensitivity is comparable to the sensitivity of the HTRU-South LOLAT survey performed with the Parkes telescope, assuming a spectral index of −2 for pulsars. Importantly, as UTMOST-2D is a transit telescope, the huge NS FoV of ∼13° would make the entire latitudinal extent of the Galactic plane probed in the SMIRF survey (|b| < 5°) fit in a single pointing for a given Galactic longitude. This means that a survey of the Galactic plane could be performed every day! Along the same line of thinking, one could perform a survey of the entire Southern sky every week. Also, similar to what is currently performed with the EW arm, the large FoV also offers the potential to monitor a massive number of pulsars (>1000) by performing commensal pulsar timing observations.
7 CONCLUSIONS
We have performed the first ever real-time multipass survey of the southern Galactic plane with the newly refurbished UTMOST telescope and reported the use of a fully automated unsupervized scheduler that is capable of intelligently observing sources/fields across the predominant observing programmes in place at UTMOST. The wide FoV and efficient scheduling permit sweeps of ≈1500 deg2 of the Southern Galactic plane to be performed in around 14 d or less, with the potential to explore a new phase space of pulsar intermittancy. We described the single pulse and periodicity search pipelines used in the survey and provided examples of known pulsar redetections. We find that the periodicity pipeline blindly recovered ∼98 per cent of the known pulsars above the limiting sensitivity of the survey. The commensal operations with the FRB search programme and the pulsar timing programmes robotically scheduled, have resulted in the detection of seven FRBs and two pulsar glitches, respectively. We report the discovery of a new (possibly highly intermittent) pulsar PSR J1659−54. Finally, we discuss future modifications to the telescope and the data analysis pipeline that could improve the survey’s efficiency.
ACKNOWLEDGEMENTS
We thank the referee for a thorough reading of the manuscript and suggesting improvements. We thank V. Ravi and R. Wharton for helpful discussions. This research was primarily supported by the Australian Research Council Centre of Excellence for All-sky Astrophysics (CAASTRO; project number CE110001020). The Molonglo Observatory is owned and operated by the University of Sydney, with support from the School of Physics and the University. MB and SO acknowledge the Australian Research Council grants OzGrav (CE170100004) and The Laureate fellowship (FL150100148). The Molonglo Observatory is owned and operated by the University of Sydney with support from the School of Physics and the University. The Laureate fellowship and the Swinburne University of Technology support the operations and upgrade of the UTMOST telescope. AD is supported by an ARC Future Fellowship grant FT150100415.
Footnotes
0–5000 pc cm−3 from late 2018 – to maximize FRB science.
The duty cycle term on the far right is ignored for single pulse estimates.
REFERENCES
APPENDIX A: THE SMIRF SCHEDULER
This section describes the SMIRF scheduler that was designed to maximize our observing cadence across different observing programmes. The scheduler is capable of automatically choosing appropriate sources/fields to observe across the observing programmes at UTMOST, namely the pulsar timing programme, the FRB search programme, and the SMIRF survey itself. The pipeline performs searches for single pulses from FRBs and pulsars in real time, periodicity searches for pulsars in real time, commensally times up to four pulsars inside the primary beam of the telescope and sends email alerts of potential candidates and problems with the observing system, if any. An overview of the core algorithm behind the SMIRF scheduler is detailed below.
A MySQL backend data base forms the backbone of the scheduler operations. The fields/sources of interest are first sorted into one of the following categories of pointings: a pulsar timing pointing (TIME), an FRB search pointing (FRB-SRCH), an FRB follow-up pointing (FRB-FOLLOWUP), or a SMIRF pointing (SMIRF). The pointings from TIME and FRB-FOLLOWUP which overlap within the primary beam of a SMIRF survey pointing are first removed from the list of sources to observe, except for special pointings such as our flux calibrator, a high DM pulsar, PSR J1644−4559. Each of the remaining pointings is first assigned a priority (Pi) to be one of |$\rm \lbrace LOW, HIGH, IMMEDIATE\rbrace$|. Each pointing also has an associated minimum dwell time(|$D_{ \rm min}^i$|), which denotes the minimum wait time between observations before a pointing starts to ‘compete’ for time again. The minimum dwell time for the SMIRF and the FRB-FOLLOWUP pointings is 4 d while the minimum dwell for the TIME pointings are decided based on their respective science cases. The millisecond pulsars J0437−4715 and J2241−5236, which are used to verify the stability of our station clock, are assigned the lowest dwell time of 2 d. The cadence for glitching and intermittent pulsars are generally higher than other pulsars. All pointings have a crucial cadence of |$C\rm _{min}=20$| d after which a pointing is given the highest priority of IMMEDIATE so that it is observed soon.
The scheduler is an object-oriented and multithreaded system designed to maximize completeness across all available pointings, mediated by the minimum cadence per pointing. Once the scheduler is started, it spawns three operational top-level threads: a status monitor thread that polls the status of the backend and the telescope position periodically, a control thread that oversees the operation, regularly monitors the priority of the pointings and sends error reports via email for any problems with the telescope operation and a core thread that handles the scheduling and observations. Fig. A1 provides a (highly) simplified schematic of the scheduler operation. First, the scheduler checks for and chooses any user ‘requested’ pointing that needs to be observed. Any pointing that ‘must’ be observed on a given day, can be specified up to 24 h in advance, via a TCP socket. Such requested pointings are assigned the highest priority.
A schematic of the SMIRF scheduler. See Appendix for details.
Secondly, the scheduler also performs automatic phase calibration observations, which can be enabled when the scheduler is initialized. If there has not been phase calibration performed in the previous 48 h and a potential calibration quasar is about to transit the primary beam, this procedure is assigned the second highest priority.
Thirdly, the scheduler checks for any pointing that has ‘IMMEDIATE’ priority. Any pointing that has crossed |$C\rm _{min}$| days since the previous observation, is dynamically provided ‘IMMEDIATE’ priority by the control thread. If there are no such pointings, the scheduler then runs its the core scheduling branch.
The branch starts by obtaining the pointing that is nearest in Hour Angle (HA) to the present telescope position which has been observed the least number of times among other ‘competing’ pointings. If this least-observed pointing (LOP) is outside the primary beam of the telescope, an appropriate ‘detour’ is chosen such that the telescope observes an ‘interim’ pointing in the meantime. Such an interim pointing is chosen by collating all the pointings that are in between the LST (local sidereal time) and the RA of the LOP. A first shortlist is made by choosing only the pointings which can be observed with sufficient time to get back to the declination of the LOP. Among the shortlisted pointings, the best interim pointing is chosen by performing a multilevel sorting algorithm that provides a reasonable trade-off between minimum slew time, maximum time since the previous observation of the pointing, and its observing priority in the data base. In the extremely rare case where there is no pointing to go to, an FRB search is performed at the present declination of the telescope.
Once a pointing is chosen, the scheduler instructs the telescope to slew to the required declination. A ‘wait-time’ FRB search observation is performed if the telescope is waiting for the pointing to reach a minimum hour angle, usually at the quarter-power point of the primary beam in the east–west direction. During all observations barring phase calibration, real-time searches for pulsars and FRBs are performed as detailed in the main text. Once the observations are complete, a post-observation manager handles what needs to be done with the data. The manager calls the appropriate subroutines for phase and flux calibration observations or starts the periodicity search pipeline in case of SMIRF pointings. In case there is a problem, either with the telescope drives or the backend system, the status monitor thread will instruct the control thread to cease operations. The control thread then performs a fail-safe shutdown of several parts of the system and sends an error report as an email to the observing team.
