(AGENPARL) - Roma, 29 Febbraio 2024 - (AGENPARL) – gio 29 febbraio 2024 Quaderni FinTech
Dimensionally reduction
techniques to support
insider trading detection
Consob – Scuola Normale Superiore di Pisa
febbraio 2024
Nella collana dei Quaderni FinTech
sono raccolti lavori di ricerca relativi
al fenomeno «FinTech» nei suoi molteplici aspetti
al fine di promuovere la riflessione e
stimolare il dibattito su temi attinenti
all’economia e alla regolamentazione
del sistema finanziario.
Comitato editoriale
Paola Deriu (coordinatrice)
Valeria Caivano
Daniela Costa
Monica Gentile
Paola Soccorso
Segreteria di redazione
Eugenia Della Libera
Tutti i diritti riservati.
È consentita la riproduzione
a fini didattici e non commerciali,
a condizione che venga citata la fonte.
CONSOB
00198 Roma – Via G.B. Martini, 3
20121 Milano – Via Broletto, 7
h www.consob.it
Tecniche per la riduzione dimensionale dei dati
a supporto del rilevamento dei casi di insider trading
Consob – Scuola Normale Superiore di Pisa
Sintesi del lavoro
L’identificazione degli abusi di mercato è un’attività estremamente complessa che richiede l’analisi di insiemi di dati grandi e complessi. Lo studio propone un metodo di apprendimento automatico non supervisionato per il rilevamento di anomalie contestuali, che fornisce un
supporto alla vigilanza sui mercati finalizzata all’identificazione di potenziali attività di insider
trading. Nello specifico, lo studio – basato su un data set anonimizzato – affronta il problema di
identificazione di potenziali casi di insider trading e propone un metodo diverso rispetto ai precedenti studi che hanno fatto uso di tecniche di unsupervised machine learning: in questo caso,
infatti, viene applicata la tecnica di decomposizione e successiva ricostruzione di una serie temporale di dati attraverso l’analisi delle “componenti principali” (PCA, Principal Component Analysis) e l’uso di autoencoders, in relazione alle posizioni assunte da gruppi di investitori in un determinato titolo azionario in prossimità di un evento price sensitive. L’unico input del metodo è
la posizione di trading di ciascun investitore attivo sull’asset per il quale si è verificato un evento
price sensitive (PSE). Dopo aver determinato gli errori di ricostruzione relativi ai profili di profili
di trading, vengono imposte diverse condizioni al fine di identificare gli investitori il cui comportamento potrebbe essere sospetto di insider trading in relazione al PSE. In termini intuitivi, la
logica che viene seguita nella procedura di identificazione di comportamenti anomali da parte
degli investitori considera la posizione media ricostruita attraverso la tecnica PCA come rappresentativa di un’operatività normale. Qualsiasi scostamento nell’operatività di un singolo investitore dal comportamento medio ricostruito nel periodo di osservazione (che sia superiore ad una
certa soglia di sensitività) viene segnalato dall’algoritmo come anomalo e potenzialmente meritevole di approfondimenti ulteriori attraverso tecniche di indagine “tradizionali”. Un risultato
particolarmente significativo di questo studio è la soddisfacente convergenza dei risultati ottenuti
con quelli derivati dall’applicazione delle tecniche di unsupervised machine learning descritte nel
precedente paper “A machine learning approach to support decision in insider trading detection”,
anch’esso frutto della collaborazione tra l’Istituto e la Scuola Normale Superiore di Pisa.
Il presente lavoro, frutto della collaborazione tra la Consob e la Scuola Normale
Superiore di Pisa, è stato curato da:
Adele Ravagnani – SNS Pisa
Fabrizio Lillo – SNS Pisa, Dipartimento di Matematica, Università di Bologna
Paola Deriu – Consob
Piero Mazzarisi – SNS Pisa, Dipartimento di Economia Politica e Statistica, Università di Siena
Francesca Medda – Consob
Antonio Russo – Consob
Dimensionality reduction techniques to support
insider trading detection1
Adele Ravagnani2, Fabrizio Lillo3, Paola Deriu4,
Piero Mazzarisi5, Francesca Medda6, Antonio Russo7
January 8, 2024
Abstract
Identification of market abuse is an extremely complicated activity that requires
the analysis of large and complex datasets. We propose an unsupervised machine
learning method for contextual anomaly detection, which allows to support market
surveillance aimed at identifying potential insider trading activities. This method lies
in the reconstruction-based paradigm and employs principal component analysis and
autoencoders as dimensionality reduction techniques. The only input of this method
is the trading position of each investor active on the asset for which we have a price
sensitive event (PSE). After determining reconstruction errors related to the trading
profiles, several conditions are imposed in order to identify investors whose behavior
could be suspicious of insider trading related to the PSE. As a case study, we apply
our method to investor resolved data of Italian stocks around takeover bids.
Keywords: Dimensionality reduction, Principal component analysis, Autoencoder,
Insider trading, Market abuse, Unsupervised learning.
This paper represents the personal opinions of the authors and does not bind the membership organization in any way.
Bologna, Italy.
e la Borsa, Italy.
Contents
Introduction ……………………………………………………………………………………………………………… 2
1.1. Literature review …………………………………………………………………………………………………. 2
Method …………………………………………………………………………………………………………………………. 5
2.1. Overview ……………………………………………………………………………………………………………….. 5
2.2. Dimensionality reduction methods …………………………………………………………………… 7
Data …………………………………………………………………………………………………………………………….. 11
3.1. Transaction reporting database………………………………………………………………………. 11
3.2. Price sensitive events database ………………………………………………………………………. 12
Results ……………………………………………………………………………………………………………………….. 13
4.1. Other case studies …………………………………………………………………………………………….. 21
Conclusion ……………………………………………………………………………………………………………….. 23
References …………………………………………………………………………………………………………………………. 25
Appendix
A The choice of K in PCA ……………………………………………………………………………………….. 27
B PCA on data in two different formats ……………………………………………………………… 28
C Relation between linear autoencoders and PCA……………………………………………… 30
D Anomaly detection with autoencoders …………………………………………………………….. 32
E Households versus firms ……………………………………………………………………………………… 35
Introduction
Insider trading is the unlawful practice of trading by exploiting nonpublic confidential
information about a listed company. It is a type of market abuse: it prevents full and
effective market integrity, it violates natural demand-supply dynamics, it compromises
public confidence. Knowing in advance how the price will likely move in response to the
release of confidential information to the market, i.e. price sensitive event (PSE), such
as, for example, the announcement of a takeover bid, can be easily exploited to make a
profit. Such a type of practice is prohibited or criminalized in most jurisdictions around
the world [4]. However, rules are specific of each country and efforts in persecuting
insider trading vary considerably. In the European Union, it is expressly prohibited
and administratively sanctioned. The member states are left with the possibility of
also imposing criminal sanctions.
The “proof” and the subsequent imposition of a sanction (either administrative or
criminal) to an investor that has operated as an insider is however a complex process,
involving many steps: (i) the detection of alerts pointing to anomalies that appear
attributable to abusive behaviors, (ii) the concrete assessment of the allegedly suspicious conduct with respect to possible rationale that may have supported the strategy
under analysis, (iii) the investigation phase aimed at gathering evidence and clues of
the abusive conduct, and (iv) the subsequent legal trial to confirm the fact that the
unlawful conduct was committed.
In [19], we focus on the first two steps. We propose a methodology, based on unsupervised machine learning techniques, that is capable of providing an indication on
whether the trading behavior of an investor or a group of investors is anomalous or not,
thus supporting the monitoring and surveillance processes by the competent Authority
and the assessment of the conduct. Our previous approach combines two methods,
which are both based on well-known techniques, the k-means clustering algorithm [12]
and the statistically validated co-occurrence networks [24]. With this new work, we
want to provide an extension of the first method of [19]. The latter aims at identifying investors with suspicious behavior related to a price-sensitive event, by means of
a dynamic clustering approach. This is done by focusing both on the discontinuities
in the trading activity of single investors with respect to their normal activity and to
the behavior of their peers, i.e., investors with similar activity. The first step of this
method is the characterization of the trading activity of each investor in several time
windows. Focusing on a time window, each investor is associated with a point in a
three-dimensional space, which corresponds to three trading features (signed turnover,
magnitudo, maximum exposure) that are relevant to our insider trading detection task
and summarize the activity of the investor in that time window. The choice of three
features was motivated by explanability reasons, since we can have a graphical representation and more easily interpret our findings. In this new work, we aim to overcome
this choice of the three trading features by employing a dimensionality reduction approach. The idea is that the model should identify the features more relevant to
investors’ characterization by itself.
Literature review
Our work suits into the framework of anomaly detection. This field has been widely
explored in the literature, especially in the last years, when its developments have been
going at the same pace as machine learning. The applications in the field of financial
fraud detection are numerous [25] and, among them, some works are related to the
detection of market abuse such as [21, 8, 17, 22]. However, insider trading detection in
stock markets is a fairly unexplored topic.
Anomaly detection basically consists in identifying data instances that cannot be
associated with normal behavior and that are rare in the data set. The goal of anomaly
detection is to define a region of the features’ space where normal observations lie;
observations that do not lie in this region are defined as anomalies [5]. Identifying this
normality region is not straightforward: the boundary between normal and anomalous
behavior is not always sharp, behaviors that are actually anomalous could be disguised
in order not to be identified, the definition of normal behavior could be time varying
and it is strongly dependent on the application domain, it is difficult to distinguish
noise from anomalous behavior [5]. From a practical point of view, there are four main
aspects which determine the formulation of the anomaly detection method: availability
of data labels, the desired output of the technique, the nature of the input data, the type
of anomaly. A different type of anomaly detection approach is employed depending on
the availability of data labels: supervised when each observation is labeled as normal
or anomalous, semi-supervised when training data do not contain any anomalies and
unsupervised when no labels are provided as in our interest case. Typically, the output
of the anomaly detection algorithms associates with each observation a score, which
quantifies the magnitude of its anomalous character. Setting a suited threshold, the
ranked list of anomalies can provide labels for each data instance. Data instances
can be of various type (binary/categorical/continuous, univariate/multivariate) and
independent among them or related to each other, as it is the case of time series and
sequences, spatial data and graph data, for which ad hoc methodologies have to be
employed [2, 1]. Concerning the type of anomalies, the standard case is represented
by point anomalies, which are single elements identified as anomalous; they could be
global or local depending on whether the entire feature space or a specific region of
it is considered [10]. Interestingly, there are cases where an element can be seen as
normal, but when a given context is taken into account, it turns out to be an anomaly.
We refer to this type as contextual anomalies [5], also termed conditional anomalies
[23]. It may happen, for instance, that an investor has operated on a stock and,
without a context, such an operation looks similar to other operations in the market.
However, when compared to the own past behavior of the investor or to the operations
of other investors, some discontinuity or synchronization patterns may be revealed and
the operation could turn out to be identified as anomalous. Contextual anomalies
problems can be tackled by algorithms for point anomaly detection once the context
is included as a new feature. Finally, we could have data instances that are normal
if considered individually, while they are anomalous together: they are the so-called
collective anomalies [5] and can occur in a data set where data instances are dependent.
It is evident that anomaly detection problems are challenging, especially in the
unsupervised setting. A variety of different approaches have been developed to address
them. In particular, the main paradigms in time-series are: clustering-based, distancebased, reconstruction-based and forecasting-based [20]. Among them, the methods are
multiple and their formulations are case-by-case dependent.
In this work, we develop an approach which aims at identifying contextual anomalies and lies in the reconstruction-based paradigm [20]. This framework aims at training
models that reconstruct normal data instances well. In this way, we expect that anomalous data will be reconstructed with a large error. An anomaly is detected when the
reconstruction error is greater than a threshold i.e.
ˆ >?
||X ? X||
ˆ is its reconstructed counterpart and ? a suitable
where X is an original data sample, X
threshold. Models’ performances are usually compared in terms of the most common
metrics e.g. precision, recall and F1-scores [20].
Our case is even more complicated since we are not provided with labels which
allow to compute the metrics to assess the models’ performances. Therefore, in order
to compare the results of different reconstruction models, we have to rely on qualitative
inspections.
The standard model employed in reconstruction-based approaches is Principal Component Analysis (PCA) [14]. Its goal is to obtain a compressed representation of data,
retaining the most important features. Data are mapped to a lower dimensional space
by orthogonal transformations that aim at maximizing data variance or equivalently,
minimizing reconstruction error.
The nonlinear counterpart of PCA is an autoencoder [11]. As PCA, autoencoders’s
goal is to minimize reconstruction errors but, in this case, the compression and decompression steps are made by means of neural network layers. Complex autoencoder
architectures can be devised, as deep, convolutional, LSTM, variational autoencoders
[11]. Also, generative adversarial networks have been used in reconstruction-based
methods [3]. Moreover, combined approaches as in [6] can be employed.
If an autoencoder is provided with one hidden layer and linear activation functions,
the analogy with PCA is evident and in the literature, it has been investigated in
several works. In particular, in [16], the authors characterize the loss landscapes of
linear autoencoders (LAEs), prove that LAEs with L2 regularization learn the PCA’s
principal directions and provide an algorithm to recover them from LAEs’ results.
Contributions of the paper and outline1 The main contributions of the paper
can be summarized as follows:
• We devise a method to support decision in insider trading detection which is not
based on the definition of specific trading features;
• Our method is an unsupervised approach for contextual anomaly detection, without any labels to check results and compare performances;
• We apply principal component analysis and autoencoders for the reconstruction
of trading profiles.
The paper is organized as follows. In Section 2, the proposed method is described.
Section 3 presents the data set we use in our empirical analysis and Section 4 presents
the results obtained by our method, with a special focus on one PSE. Finally, conclusions are drawn in Section 5. In the appendix, some figures, which are explained in the
main text, are reported, and other collateral issues are investigated.
The methodology presented in the paper was conceived in 2023 for the purpose of developing a proof
of concept. It is, in no way, a tool used in the analysis and investigations carried out by Consob. The
methodology may possibly constitute the future one of the tools to help and support the preliminary analysis
and detection activities more efficiently. Any subsequent enforcement activity will, in any case, be based on
the broader set of information that is gathered in the course of investigations and other possible types of
analysis.
Method
Overview
As in [19], we are tackling an unsupervised problem without any availability of labels
and we consider a specific class of price sensitive events (PSEs), namely announcements
of takeover bids. A takeover bid is a public offer made by a physical person or a legal
entity who is willing to buy other shareholders’ shares at a price higher than the stock
market value. If investors know in advance when the announcement of the takeover
bid will occur, they can exploit their information by buying before the PSE. Indeed,
when the takeover bid occurs, the shares’ price goes up aligning with the offer price
and thus, the informed investors can sell by making a no-risk profit.
We focus on a single asset, the one for which we have a PSE, and on a time window
with T trading days. The first part of this time window – e.g. 6 months – is a reference
period and the second part – e.g. 1 month – is an investigation period i.e. a short time
window preceding the PSE that will be defined as ? in the following. As a first step,
we compute the trading position of each investor on each day. Given N0 investors and
T trading days {t0, t1, . . ., tT }, the position of investor i on day t is defined as follows:
xi (t) =
[Vb (i, t? ) ? Vs (i, t? )]
t0 ?t? ?t
where Vb/s (i, t) is the number of shares bought/sold by investor i on day t. Therefore,
a vector xi = xi (t0 ), . . ., xi (tT ) ? RT is assigned to each investor. As usually done,
we normalize data as
xi (t)
xi (t)
xi (t) ?
maxt |xi |
||xi ||?
and investors with constant positions i.e. investors who do not trade or are strict daily
investors (i.e. the number of shares purchased and sold on each day are equal), are
discarded.
In the definition of Equation 1, we assume that investors’ positions are null on
t0 . Of course, this is not true in general. However, since information on the precise
composition of the portfolio of each investor is not available, this sounds as the best
proxy of asset positions. In the following, we will see that actually this is not an issue
for this new method, contrary to what happens for our previous method [19].
We also observe that positions are computed using the number of shares and not
Euro. The reason is that the monetary value of a portfolio fluctuates in response to
the changing price and these fluctuations affect in the same direction positions with
the same sign (e.g. long or short). Thus, spurious correlations between positions might
be detected when using Euro as a unit of measurement.
Indicating with N the number of investors with non-constant positions, we end up
with a data set X ? RN,T with rows xi . This data set is the input of a dimensionality
reduction method (we are going to use PCA and autoencoders), which will allow to
obtain a reconstructed representation of the data after a compression i.e.
ˆ = f2 (Z) where X, X
ˆ ? RN,T, Z ? RN,K
X ? Z = f1 (X) ? X
and such that the reconstruction error is minimized with respect to the transformations
f1 and f2 i.e.
ˆ = arg min ||X ? X
ˆ ? ||2F
ˆ?
where || · ||F is the Frobenius norm.
In the compression phase, observations are mapped to a lower dimensional space
that captures common and essential characteristics. In our setting, the features which
are subjected to compression are the positions of investors’ on each day. This consists in identifying a subset of days or combinations of them with a major role in the
characterization of agents’ trading activity.
After the dimensionality reduction step, investors with anomalous activity (potential insiders) are identified following the reconstruction-based paradigm and assuming
they are substantially less numerous than investors with normal behavior. The preparatory steps of the method we develop are the following:
• compute the reconstruction errors
?i (t) = |xi (t) ? xˆi (t)| i = 1, . . ., N, t = 1, . . ., T,
we expect that normal observations have low ?i, while anomalies have large ?i ;
• compute the anomaly scores i.e. the largest reconstruction errors for each investor:
s?i = max ?i (t) i = 1, . . ., N ;
• localize the largest reconstruction errors:
t?i = arg max ?i (t), i = 1, . . ., N
which are such that
?i (t?i ) = s?i, i = 1, . . ., N ;
• compute nt = card{i : t?i = t} ?t = 1, . . ., T, where nt is the number of investors
having the largest reconstruction error on day t;
• compute di for i = 1, . . ., N i.e. the number of activity days of each investor.
Finally, in order to detect potential insiders, we devise a method which is based on
the following idea. In order to be anomalous, an investor should satisfy the following
conditions: (1) to have at least one day for which the reconstruction error is large,
(2) the corresponding time lies in the investigation period, (3) she has either a small
number of activity days or her identified anomalous score is on a day when not too
many other investors do, and (4) she is in a net buying position on the day of the PSE.
This last condition can be set by imposing that the difference between the position
on the PSE and the position on the first day of the reference period is larger than a
threshold, that we choose equal to 0.5.
Formalizing the above conditions, we say that an investor i is anomalous if
?i (t), t, nt : ?i (t) ? ??, t ? ?,
nt d? or ?nt if di ? d?,
i has a net buying position on the PSE
?= ?.
This criterion depends on three threshold parameters d?, ??, and n? . The parameter
d?, the minimal number of days in item (3) of the criterion above, is set to 3. Instead,
we choose ?? and n? in a data driven fashion: we estimate the probability density
function of the anomaly scores s?i and of the times of the largest reconstruction errors
t?i . Since we observe that the former distribution is bimodal (see the left panel of Figure
3), we expect that normal investor profiles are associated to small anomaly scores s?i,
while anomalous ones to high scores. Thus ?? is chosen as the local minimum between
the two modes of the distribution. In practice, we relied on the module signal of the
Python library scipy. Finally, n? is chosen as the top decile of its distribution.
Supervising authorities are often interested in a ranking of potential insiders to
identify the most suspicious investors. Our approach is able to deliver such a ranking.
Investors are mapped in a two-dimensional space (s?i, n
¯ t?i ), where
¯ t?i = nt?i I[di > d? ]
and then the two features are normalized to take values in [0, 1]. The Euclidean distance
between each investor and the point (1, 0) is the metric for our ranking. The smaller
the distance the higher the ranking.
Finally, it is important to point out the extreme unsupervised nature of our problem. We are not provided with labels associated to each investor so, we train models
for dimensionality reduction by using all data and we cannot compare models’ performances in terms of accuracy. This and our previous work [19] tackle the same issue
and we could be tempted to employ [19]’s results as ground truth. However, with this
dimensionality approach, we aim to provide a new method that could give another tool
to support regulators’ investigations related to insider trading detection. Therefore,
the results of [19] are not validation data: in this new work, they are employed for
comparisons and robustness checks.
Dimensionality reduction methods
Principal Component Analysis
A standard method to apply dimensionality reduction is Principal Component Analysis
(PCA) [14]. Starting from a feature scaled data set X, the goal of PCA is to obtain a
ˆ by means
compressed representation ZK of data and then, a reconstructed version X
of orthogonal transformations:
ˆ = ZK PK
ˆ ? RN,T, ZK ? RN,K, PK ? RT,K,
X ? ZK = XPK ? X
where X, X
and the transformation matrix PK is such that the reconstruction error is minimized
with a rank constraint, i.e.
ˆ = arg
ˆ ? : rank(X
ˆ ? )?K
ˆ ? ||2F .
||X ? X
The solution is the truncated Singular Value Decomposition (SVD), as follows from
the Eckart-Young theorem [9] or analogously, it is obtained by applying the spectral
theorem on the covariance matrix of X which is mean-centered:
Cov(X) =
X X = P ?P T
where P ? RT,T is orthogonal (P T = P ?1 ), ? ? RT,T and ? = diag(?1, ?2, . . ., ?T )
with ?1 > ?2 > . . . > ?T . The sum of the eigenvalues ?1, . . ., ?T of the covariance
matrix is the total variance of the data. Thus, keeping more components means being
able to explain more data variability. The eigenvectors matrix P is defined as P =
[p1, p2, . . ., pT ] and pi, i = 1, . . ., T are the loading vectors or principal components.
The dimensionality reduction with K components is obtained as
ZK = XPK
ˆ = ZK P T and so X
where PK = [p1, . . ., pK ] ? RT,K, the reconstructed data as X
XPK PK . More explicitly, we have
ˆi (t) =
(xi · pk )pk (t) i = 1, . . ., N, t = 1, . . ., T.
It is evident that PCA is a decorrelation transformation and that its solution is not
unique. Indeed, the loss is invariant under the transformation P ? P U, where U is
any orthogonal matrix. Under this transformation, the loading vectors are transformed
into a different orthonormal basis for the same subspace. Moreover, according to the
Eckart-Young theorem [9], the truncated SVD is the best low-rank approximation of a
matrix. Therefore, PCA is the linear dimensionality reduction method that minimizes
the least squares error in the distortion when we project back to the original space2 .
Finally, it is worth noticing that, compared to other insider detection methods, the
one based on PCA does not depend on the knowledge of the initial position of the
investors. As explained in Subsection 2.1, this information is indeed lacking in our
dataset and in general it can be difficult to obtain because it requires the knowledge
of the whole past history of an investor trading activity. Clustering based on k-means,
adopted in [19] depends instead on the arbitrary choice of the initial position of the
investors. To show that this is not the case for PCA, we prove that the PCA reconstructed position error ? is invariant under the addition of an arbitrary constant to the
vector of positions of a given investor. Proof. Let us consider the vector xi describing the position of investor i and let us add an arbitrary constant C to it, obtaining
ˆi and x
i = xi + CIT x1 . Denoting with x
i the reconstructed positions, it is direct to
show that xˆi = x
ˆi +CIT x1 . First, we observe that, given the high number of investors,
performing PCA on a dataset where investor i has position xi and then, on a dataset
where investor i has position xC
i leads to loading vectors which are basically the same.
Observe that our starting data set is in the format N × T, i.e. the trading days are the features
that are subjected to compression. Alternatively, we could start with a data set Y ? RT,N where the
features are the investors. In this case, PCA consists in identifying a subset of investors, or combinations
of them, with a major role in the characterization of agents’ trading activity. However, this would lead to
a more time consuming and computationally expensive procedure. Indeed, we would need to obtain the
eigendecomposition of the covariance matrix Cov(Y ) ? RN,N, which is much larger than Cov(X) ? RT,T
since N ? T . Furthermore, as shown in Appendix B, if the same feature scaling is applied on the data, the
results we obtain starting from the data set in two different formats are analogous.
Referring to Equation 4,
i (t) =
i · pk )pk (t) =
[(xi + CIT x1 ) · pk ]pk (t) =
(xi · pk )pk (t) + C
(IT x1 · pk )pk (t) =
= xˆi (t) + C [p1 (1)p1 (t) + p2 (1)p2 (t) + . . . + pK (1)pK (t)] + . . . +
+ [p1 (t)p1 (t) + p2 (t)p2 (t) + . . . + pK (t)pK (t)] + . . . +
+ [p1 (T )p1 (t) + p2 (T )p2 (t) . . . + pK (T )pK (t)] =
= xˆi (t) + C[(PK PK
)1t + . . . + (PK PK
)tt + . . . + (PK PK
)T t ] =
?t = 1, . . ., T
= xˆi (t) + C
where (PK PK
lm is the element l, m of the matrix PK PK and in the last step we
exploit that the matrix PK is orthogonal. As explained in Subsection 2.1, our anomaly
detection approach is reconstruction-based and ?i = ?C
i indeed,
ˆi C ? xC
xi + CIT x1 ? xi ? CIT x1 || = ?i .
i = ||x
i || = ||ˆ
Therefore, the reconstruction error is independent of C and if the profile xi is identified
as anomalous, the same will be true for xC
i . ?
However, in our approach, after trading positions are computed, they are normalized
according to Equation 2 and so, computing the position of an investor setting the
zero of her portfolio on a different day means that the arbitrary constant C is added
to her unnormalized position. Let us define as ?i the position of investor i before
normalization. Then, we define ?iC = ? + CIT x1 as the position of the same investor
computed by setting the zero of the portfolio on another day. After normalization, the
positions are
?i (t)
||?i ||?
?iC (t)
?i (t) + C
||?i + CIT x1 ||?
||?i ||?
xi (t) =
Therefore, we have that the reconstructed positions of xC
i are
i (t)
?i + CIT x1
· pk pk (t) =
||?i + CIT x1 ||?
IT x1
· pk pk (t) + C
· pk pk (t) =
||?i + CIT x1 ||?
||?i + CIT x1 ||?
||?i ||?
· pk pk (t) +
||?i + CIT x1 ||?
||?i ||?
||?i + CIT x1 ||?
||?i ||?
ˆi (t) +
||?i + CIT x1 ||?
||?i + CIT x1 ||?
?t = 1, . . ., T,
and the reconstruction error is
This implies that ?C
i = ?i if
||?i ||?
?i .
||?i + CIT x1 ||?
||?i ||?
||?i + CIT x1 ||?
which holds if maxt |?i (t)| = maxt |?iC (t)| given that maxt |?iC (t)| =
? 0. As we will show
in Section 4, this last condition holds for the majority of the profiles in our dataset.
Autoencoders
PCA is a linear method, consisting in applying the loading vectors’ matrix to the
starting data twice. Its nonlinear counterpart is an autoencoder (AE)[11]. Autoencoders’ goal is analogous to PCA’s: starting from a data set X, they aim to obtain a
compressed representation of data Z and then, a reconstructed version X:
ˆ = f2 (Z) where X, X
ˆ ? RN,T, Z ? RN,K .
X ? Z = f1 (X) ? X
The transformations f1 and f2 are such that the reconstruction error is minimized i.e.
ˆ = arg min ||X ? X
ˆ ? ||2
ˆ?
and, in this case, the compression and decompression steps are made by neural network
layers. For an AE with one hidden layer, we have
ˆi (t) = g2
g1 (xi · W1 (:, k))W2 (k, t) i = 1, . . ., N, t = 1, . . ., T,
RT,K,
where W1 ?
W2 ? RK,T are layers’ weight matrices, also called encoder and
decoder, and g1, g2 are activation functions. The layers’ weight matrices are determined
by common gradient descent algorithms like Adam [15].
Above, we focus on an autoencoder with one hidden layer in order to highlight the
analogy with PCA. Indeed, as it is shown in [16], if the activation functions are linear,
the autoencoder with L2 -regularization learn PCA’s principal directions. This issue
is examined in depth in Appendix C. However, autoencoders can be deeper and can
have complex architectures such as the well-know convolutional, LSTM, variational
autoencoders [11].
As we saw above, the solution of PCA follows from the Eckart-Young theorem [9].
The latter states that the solution to the problem
ˆ ? ||2
||X ? X
ˆ ? : rank(X
ˆ ? )?K
is given by the truncated SVD. The latter approximates excellently data with linear
relationships. On the other hand, concerning the reconstruction of nonlinear data,
autoencoders outperform PCA, as empirical results in different fields show e.g. applications on image reconstruction. Theoretically, the difference between the problems
ˆ For PCA we impose
tackled by PCA and AE is the constraint on the rank of X.
ˆ ? K, so there are no more than K columns of X
ˆ which are linearly indepenrank(X)
dent. Equivalently, we have no more than K independent features. On the other hand,
AEs aim at minimizing the reconstruction error without any constraint on rank(X).
ˆ ? (K, T ] so, more independent features.
This means we could end up with rank(X)
Pros and cons of the different methods
PCA should be preferred against AEs if small datasets are considered, more interpretability and nested solutions are needed. PCA is also easier to implement than
AEs. Moreover, it needs less computational resources and less training time.
The interpretability of the results which are provided by PCA, is due to the linearity
of the method. However, this linearity can be a downside if data are nonlinear. On the
other end, AEs’ nonlinearity allows to capture complex relationships in the data. This
last bright side of AEs leads to a better performance in the reconstruction of outliers,
compared to PCA, and this could become a downside for our anomaly detection task,
if anomalies are reconstructed such that they are indistinguishable from normal data
instances.
Finally, PCA loading factors are ordered such that the associated eigenvalues are
in decreasing order: ?1 ? ?2 ? . . . ? ?T . Thus one can measure the importance of a
factor in explaining the data and rank the factors according to this criterion. On the
contrary, the weight vectors learned by AE are not constrained to form an orthonormal
basis, nor to have a meaningful ordering.
Choice of K
The choice of the dimension K of the compressed representation of the input data
should achieve a trade-off between capturing enough information and avoiding overfitting, which could lead to reconstruct profiles of investors with anomalous behavior
well.
The relying assumption of our use of the dimensionality reduction approach is that
the essential and common characteristics are captured by the lower dimensional space
and that they explain a large fraction of data variance. Then, anomalous behavior cannot be reproduced given the compression and decompression, and anomalous
observations have higher reconstruction errors than normal ones. However, the unsupervised nature of our case makes it extremely complicated, because our training data
are anomaly-contaminated.
As a starting point we rely on standard methods to set the parameter K, like the
percentage of explained variance and the Scree plot, that is the plot of the eigenvalues
as a function of K [14]. However, given their erratic nature, we perform an analysis,
which helps us in the choice. Let us define AK as the set of investors identified as
anomalous when K is used as dimension of the latent space. First, we determine the
cardinality of AK as a function of K. Then, we study the stability of this set by
computing the Jaccard similarity [13] between each AK and AK?1 . K is set to the
lowest value of the interval in which we have stability in our results.
Transaction reporting database
The analysis is based on transaction reports collected by Consob for the Italian stocks,
according to the directive 2014/65 by European Union, also called MiFID II3 . The
In a nutshell, the MiFIDII/MiFIR regime has introduced new regulations for European financial markets
and, among them, the transaction reporting obligation that requires investment firms or intermediaries
relevant dataset was built aggregating the daily transactions of all investors operating
in any of the Italian stocks, in the period from January 1, 2019 to September 30, 2021.
In details, the dataset was built according to the following rules: i) all the information
related to the identity of individual investors have been anonymized; ii) with reference
to each stock (identified by its ISIN code), each data point keeps a record of:
1. anonymous identifier of the investor;
2. type of investors (household: H, investment firm: IF, legal entity: L);
3. trading venue of the operation (Borsa Italiana – MTA, London Stock Exchange LSE, off-exchange, etc.) for a total of 224 venues;
4. day of the operation;
5. buy and sell volumes (in shares);
6. buy and sell Euro volumes;
7. number of buy and sell contracts;
8. price of both the first and the last contracts (if there are more than one contract,
otherwise they coincide);
9. minimum and max prices of contracts (if there are more than one contract, otherwise they coincide);
10. average price of buy (sell) contracts.
In the period covered by the data set, 2,253,707 investors were observed, operating in
286 Italian stocks. This is the same data set used in our previous paper about insider
trading detection [19] and in another work related to the investigation of the trading
behavior of Italian investors during the Covid pandemic [7].
Price sensitive events database
In addition to the transaction reporting database, a data set containing several price
sensitive events (PSEs) was built; such events, obviously public, had all been analysed
by the competent Authority with the aim of market abuse detection, by means of
standard analytics methodologies. PSEs are events or a set of circumstances relating
to listed companies which, when made public, had a significant impact on the price of
the company’s shares.
Our focus is on insider dealing in the Italian Stock Exchange. Investors who know in
advance when a PSE will occur, can trade in a rewarding manner before the information
spreads, thus closing their position after the PSE and making a profit. For instance, if
a investor knows a few days before its public announcement that a takeover bid is going
to occur for a given stock, they could exploit such information by buying shares of the
stock considered. When the takeover bid occurs, the shares’ price goes up aligning
with the offer price and thus, the informed investor can sell by making a no-risk profit.
PSEs dataset contains a list of takeover bids for a number of stocks. As known, a
takeover bid is a public offer made by a physical person or a legal entity who is willing
executing transactions in financial instruments to communicate “complete and accurate details of such transactions to the competent authority as quickly as possible, and no later than the close of the following working
day”.
Stock
PANARIAGROUP
CARRARO
MOLMED
PSE date
July 28, 2020
Feb 17, 2020
Mar 31, 2021
Mar 28, 2021
Mar 17, 2020
Investigation period (?)
June 29, 2020 – July 28, 2020
Jan 16, 2020 – Feb 17, 2020
Mar 1, 2021 – Mar 31, 2021
Jan 4, 2021 – Mar 28, 2021
Dec 2, 2019 – Mar 17, 2020
Table 1: Price sensitive events. The table reports the stock name, the date of the PSE, and
the investigation period.
to buy other shareholders’ shares at a price higher than the stock market value. As
we saw, takeover bids can be exploited by an informed investor by buying before the
event. It is worth mentioning that takeover bids have prolonged effects on the market,
thus an insider can make a profit even without closing the position immediately after
the announcement.
Our data report for each PSE the stock, its date, and the time window for insider
trading investigation. This period varies depending on the type of PSE, which leads to
different definitions of the time at which an information starts to be considered price
sensitive. In Table 1, the PSEs database is displayed.
Results
As a first case study, we focus on the asset Industria Macchine Automatiche (IMA)
whose takeover bid was announced on July 28, 2020. Figure 1 shows the price dynamics
of this asset. The impact that the PSE had on the share price is evident: there is an
increase of 13.16% on the day of the announcement and the takeover bid’s price 68.0
Euro is reached. In analogy to [19], we identify the reference period as the time window
going from January 2, 2020 to June 28, 2020. Instead, the business month preceding
the PSE i.e. July 28, 2020, is the investigation period, as outlined in Table 1.
For each investor we extract from the database the asset position (in shares) at
the end of each day. We assume that the position on January 2, 2020 is zero, but, as
proved above, this arbitrary choice has no effect on the reconstruction error.
The trading days are T = 149, the investors active and with non-constant position
are N = 13, 225.
Anomaly detection with PCA
The first method we employ in order to perform the dimensionality reduction step, is
PCA. In Appendix A we show the plot of the explained variance as a function of the
number of the retained components and the Scree plot. Considering these figures we
choose the latent space dimension K = 16, which allows to retain 97% of the explained
variance. However, we perform a robustness analysis, investigating other choices of K
in Appendix A.
After a feature scaling as pre-processing step, we run our method as illustrated in
Section 2. As an example of the reconstructed trading position, in Figure 2 we show the
position of an investor compared to its reconstructed counterpart obtained by PCA.
Most of the days the reconstruction is quite close to the original trajectory and the
Figure 1: IMA price dynamics. The grey area is the reference period i.e. from January 2,
2020 to June 28, 2020. The pink area is the investigation period i.e. one business month
before the PSE.
Figure 2: Comparison between the trading position of investor PG 2081 and the reconstructed one obtained by PCA with K = 16.
Figure 3: PCA on IMA. Left. Histogram of the anomaly scores. Blue points are local
maxima, green local minima and the red point is ?? ? 0.13 that is the local minimum after
the first peak. Right. Histogram of the times when anomaly scores are observed. The inset
plot represents the histogram of the number of investors with given values of the times when
their anomaly scores are observed. The vertical dashed line is n? i.e. the top decile of the
distribution of nt .
anomaly detection method identifies the days and investors for which the discrepancy,
i.e. the reconstruction error, is large.
To identify the thresholds in the anomaly detection method, we plot in Figure 3 the
histogram of the anomaly score s?i (left panel) and of the time of their occurrence t?i
(right panel). As preannounced, a clear bimodal distribution in the former histogram
is observed. The left mode (peak) contains investors with small anomaly score, thus
“normal” investors, while the right mode contains potentially anomalous investors with
a large maximal reconstruction error. Based on this empirical evidence, we set ?? = 0.13
as the first threshold parameter to identify potential insiders.
By focusing on the histogram of the times t?i corresponding to the largest reconstruction errors (right panel of Figure 3), we observe the presence of several large peaks,
i.e. days when a large number of investors displayed a large reconstruction error. We
can understand the origin of these peaks by focusing on the largest one, happened on
April 17, 2020 when more than 1, 300 investors display the largest reconstruction error.
Looking at the price dynamics in Figure 1, we observe that on April 17, 2020 there
was a high increase (7.4%) of the share price. The large number of investors having
the largest reconstruction error on that day is likely due to their reaction to this very
volatile day4 . Clearly these peaks and the corresponding investors are not insiders and
this explains why we impose the condition on nt, that are the heights of the peaks in
the histogram of t?i, in our methodology to identify anomalous investors – see Equation
3. The parameter n? is set equal to 158, which corresponds to the top decile of the
distribution of nt, which cuts off the investors whose anomaly score falls in a peak of
t?i .
Applying our anomaly detection method, we obtain 1, 246 potential insiders out of
the 13, 225 total investors. Given the high percentage of anomalous investors that we
We remind that positions of investors are measured in shares and not in Euro, so the peaks are not
associated with change in value of a position, due to the large price variation, but to a genuine trading
activity.
obtain, their ranking, following the procedure explained at the end of Section 2.1, is
fundamental to provide more insight.
We compare our results with the findings of a method based on k-means similar
to that of our previous paper [19]. For each investor we extract the signed turnover
and the maximum exposure in the period5 and we use them as coordinates in a 2D
space6 · Then, we apply k-means to the set of points to identify clusters of investors
and we label as anomalous an agent who, in the investigation period belongs to a
different cluster than the ones in the reference period and the new cluster is the most
rewarding one with respect to the PSE. If the PSE is the announcement of a takeover
bid, the cluster with the most rewarding position is the closest to the point (1, 1) i.e.
both signed turnover and maximum exposure equal to 1. We distinguish two types of
anomalous investors. They are soft if they are active in the reference period but with a
different position than the one in the investigation period, while they are hard if they
are only active in the investigation period7 . In summary, the PCA (and later the AE)
method acts directly on the whole trading profile of each investor (thus a vector of
dimension T ), while the method based on k-means considers two features, which are
functions of the trading profile.
If we run the method based on k-means in 2D on IMA, 152 investors are identified
as soft and 705 as hard. Among the 1, 246 potential insiders identified by the method
based on PCA, 134 are soft and 671 are hard. From the comparison we find that the
first 10 ranked anomalous investors, according to the method based on PCA, are all
identified as suspicious by the method based on k-means. If we consider up to rank
50, 100, 150, 200, 300, 500, investors who are also suspicious in the framework of the
clustering method of [19] are 49, 99, 148, 185, 253, 451 respectively.
The compatibility of the two methods is a positive sign of their robustness. However
it is natural to ask what are the characteristics of the investors identified as anomalous
only by one of the two methods. Of the first 500 ranked anomalous investors, 451 are
also identified by the method based on k-means. Among the remaining ones, 46 are of
the type represented in the top left panel of Figure 4 and 3 are of the type represented
in the top right panel of Figure 4. The former performs one transaction on June 26,
2020 i.e. three days before the starting day of the investigation period. This investor
could be suspicious given her aggressive buying position just in the vicinity of the
PSE. However, this investor is not identified by the method based on k-means since
the transaction is outside the investigation period. Therefore, contrary to the method
based on k-means, the new method based on a dimensionality approach is not strictly
dependent on an arbitrary choice of the investigation period.
On the other hand, the investor in the top right panel of Figure 4 sells a portion
of her position on the day before the PSE, but still maintains a net buying position.
Given this investor was not active in the reference period, this behavior of buying and
then, selling in the investigation period, could be a strategy in order not to be identified
The signed turnover is the aggregated Euro turnover of operations within the period, with positive
(negative) sign for a net buying (selling) volume. The maximum exposure is the maximum of the absolute
value of the position in Euro turnover within the period, with positive (negative) sign if the maximum is
reached for a buying (selling) position. We refer to our previous paper [19] for a precise definition.
In [19] we consider another feature, namely the magnitudo/portfolio concentration which represents the
fraction of wealth in the investigated asset. Since our dimensionality reduction method considers only data
related to the investigated asset, in the comparison we use a k-means approach in a 2D space.
See our previous paper [19] for further details.
Figure 4: PCA on IMA. Top panels. Two anomalous investors identified by PCA but not
by k-means. Bottom left. A refers to the trading position computed setting t0 = January
2, 2019. B refers to the trading position computed setting t0 = January 2, 2020.. Bottom
right. Investor detected by the method based on k-means and not by the method based on
PCA. In all panels the vertical dashed line is the day corresponding to the beginning of the
investigation period i.e. June 29, 2020.
as suspicious by the regulator. This investor is not identified by the method based on
k-means since in the last time window there is a drop in her signed turnover which
leads the corresponding point to move away from the most rewarding cluster.
Another positive aspect of the new method based on a dimensionality reduction
approach is its ability to detect given investors as anomalous, independently of the
starting point chosen for the computation of her position (see Subsection 2.2). We
observe that the condition in Equation 5 – which states when the reconstruction error
of a given profile is the same of its shifted counterpart – holds for about 63% of investors
in our dataset, if January 2, 2020 and January 2, 2019 are two choices of zeros for the
computation of the positions. In the bottom left panel of Figure 4 we show the position
of one investor when setting to zero the position on January 2, 2020 (blue) or on January
2, 2019 (black). Interestingly, this investor is identified as anomalous by the method
based on PCA, but not by the method based on k-means. The fact that with the choice
t0 = January 2, 2019 this investor is identified as anomalous could surprise given we are
focusing on detecting insider trading related to the announcement of a takeover bid. We
recall that after this kind of PSE, the price increases and so, insiders are likely to have
positive positions before the PSE. Indeed, this investor is not identified as anomalous
Name Neurons of the hidden layers Encoding Neurons of the hidden layers
in the encoder
dimension
in the decoder
64, 32
32, 64
128, 64, 32
32, 64, 128
Table 2: Autoencoders’ architectures.
by the method based on k-means given her negative value of signed turnover. On the
other hand, if the profile is computed with the choice t0 = January 2, 2020, she has
a net buying position before the PSE and signed turnover equal to 1. Therefore, only
with this choice of t0, she is identified as soft discontinuous by the method based on
k-means.
It is also interesting to investigate why some investors are detected by the method
based on k-means and not by the method based on PCA. The total number of these
investors is 79; among them, 27 are investors with constant position in terms of shares
and so, they are not included in the analysis based on PCA. The majority of the
remaining profiles are of the type displayed in the bottom right part of Figure 4.
They are investors who have a null position in the investigation period, if positions
are computed in shares. Instead, if positions are computed in Euros, as in the method
based on k-means, the signed turnover and the maximum exposure of these investors
are equal to 1 in the investigation period. They are in the best rewarding position and
this makes them extremely suspicious according to the clustering approach of [19]. On
the contrary, these investors are not identified by the method based on PCA, since the
condition (4) relative to Equation 3 does not hold. That condition requires that the
difference between the position on the PSE and the position on the first day of the
reference period is larger than 0.5. For investors like PF 978820 (bottom right part of
Figure 4), this difference is null since the investor closes her position before the PSE.
Finally, in Appendix B, we provide a comparison between the results obtained
starting with the data set in the formats N × T and T × N . As shown in Figure 9
and 10, if data are feature scaled in the same way, there is no difference between these
results. In Appendix C, the relation between PCA and L2 -regularized autoencoders is
tested on our data set.
Going nonlinear: the Autoencoder
As extensively proved in other research fields such as image reconstruction, adopting
nonlinear and deep autoencoders can lead to a gain in performances, giving their ability
to capture more complex relations in data. However, it is important to stress that our
ultimate goal is not to best reconstruct our data. We wish to achieve a trade-off and
to avoid overfitting. The idea is to obtain a lower dimensional space which captures
the common and essential data characteristics; in this way, normal trading profiles will
be well described while anomalous ones will not.
Bearing this in mind, we investigate the use of nonlinear autoencoders for our
problem. The symmetric architectures we employ are schematized in Table 2. The
number of neurons is chosen according to the geometric pyramid rule [18] and for
Figure 5: IMA. Comparison between the trading position of investor PG 75522 and the
reconstructed ones obtained by PCA and AE-1 with K = 16.
Model
||X ? X||
132.0
168.7
137.5
129.3
121.1
97.66
96.18
97.46
97.76
98.00
s¯?
s¯? anomalous
0.4525
0.5051
0.5100
0.6088
0.4574
0.5128
0.4404
0.4997
0.4228
0.4620
s¯? normal
0.4489
0.5031
0.4536
0.4363
0.4201
0.1251
0.2099
0.1308
0.1453
0.0999
1.116
1.193
1.121
1.134
1.093
Table 3: Metrics for different dimensionality reductions
of the IMA dataset. EVS is the
V ar(X?X)
explained variance score and it is defined as 100 1 ? V ar(X) ; s¯? is the mean anomaly
score; s¯? anomalous is the mean anomaly score for investors who are identified as hard/soft by
the method based on k-means; s¯? normal is the mean anomaly score for all investors except
¯?
?s¯? normal
the ones identified as hard/soft by the method based on k-means; M1 = s anomalous
s¯? normal
s¯? anomalous
. The results related to the autoencoders are averaged over 10 runs.
s¯?
all architectures the activation function of the hidden layers is the ReLU, while the
activation function of the output layer is the hyperbolic tangent. This last choice
allows to exploit the nonlinearity of the neural networks and yet to produce outputs
with values in the interval [?1, 1], the domain of the normalized trading positions. The
loss function is the mean squared error (MSE) and Adam [15] is used as optimization
algorithm.
The PCA and the 4 autoencoder architectures of Table 2 are run on our data
set of trading positions for IMA. A comparison between the reconstructed profile of
an investor, obtained with PCA and a type of autoencoder is represented in Figure
5. Greater smoothness is associated to the profile obtained by AE-1 however, the
overall similarity between the two different profile reconstructions is evident. Table
3 summarizes the main results for all autoencoders’ architectures in terms of several
metrics. Due to their nonlinear character, we expect deep autoencoders can capture
more complex features in data. This could lead to the identification of characteristics
for the data compression which are more relevant than the ones identified by PCA,
which is a linear model. However, we need at least 5 hidden layers to outperform PCA
ˆ F . The lower MSE is accompanied by greater
in terms of the loss function ||X ? X||
explained variance score (EVS) which is defined as8
V ar(X ? X)
EV S = 100 1 ?
V ar(X)
This means that deeper autoencoders can explain a larger variance of the data.
Nonetheless, we are not solely interested in better reconstruction errors. A larger
gap between the errors of the anomalous investors and the ones who are not anomalous
is desired. Given our unavailability of labels, we compare the mean anomaly score of
investors who are detected as hard/soft by the method based on k-means [19] i.e.
s¯? anomalous, and the mean anomaly score of the other investors i.e. s¯? normal .
The two metrics M1 and M2, that we introduce, shed light on this issue. They are
defined as
s¯? anomalous ? s¯? normal
s¯? normal
s¯? anomalous
s¯?
We obtain that the model AE-1 has the greatest values of M1 and M2 i.e. it leads
to a greater gap between the anomaly scores of our proxy of anomalous investors and
the others. On the other hand, AE-4, which allows to obtain the lowest error in the
data reconstruction, leads to the lowest value of the two quantities. It is important to
stress that the comparison between the values of M1 and M2 that are obtained with
different models, provide information which could be useful to our insiders detection
task. However, the anomaly score is not the only quantity which determines whether an
investor is identified as anomalous; the distribution of the times corresponding to the
largest reconstruction errors also plays a fundamental role, both on the identification
and on the ranking. We will investigate this issue in the following by employing both
AE-1 and AE-4 for the anomaly detection step.
Anomaly detection with autoencoders
We perform our anomaly detection task by employing two different architectures of
autoencoders i.e AE-1 and AE-4. The main results are summarized in Table 4 and
compared with the results of PCA.
First of all we notice that when considering the first 150 ranked investors, the
different methods provides almost identical set of anomalous cases. This, once more,
indicates that machine learning methods (k-means, PCA, autoencoders) essentially
Notice that the EVS is different from a common metric that is usually employed for PCA, that is the
explained variance ratio (EVR), defined as
var(zk )
EV R = Pk=1
t=1 var(xt )
where xt ? RN are the columns of X ? RN,T, that is the data matrix, and zk ? RN are the columns of
ZK ? RN,K, that is X’s representation in the K-dimensional latent space. The EVR is not a good indicator
for autoencoders since it strictly depends on the activation functions we choose. Therefore, EVS is preferred
to EVR, thus allowing a fair comparison with the PCA results.
Method
|A| I/|AKM | I10
1,246 805/857 10
1,502 812/857 10
1,325 807/857 10
Table 4: IMA: anomaly detection. A is the set of investors identified as anomalous. I is
defined as I = |A?AKM | where AKM is set of investors identified as hard/soft by the method
based on k-means. In is defined as In = |An ? AKM | where An is the set of the first n ranked
anomalous investors.
agree in the identification of the most suspicious investors, demonstrating an overall
robustness of the adopted methodologies.
When we focus on the first 500 ranked investors, PCA is the method with the largest
overlap with the method based on k-means. This cannot be explained by the anomaly
score values since, as shown in Table 3, the value of the metrics M1, M2 obtained by
PCA are lower than the one obtained by AE-1. The two different dimensionality reduction methods lead to different histograms of the times corresponding to the largest
reconstruction errors, which are shown in the right panels of Figures 3 and 14. While
the histogram obtained with PCA has a maximum on April 17, 2020, the one obtained
with AE-1 has the highest peak on the day of the PSE. Contrary to PCA, the autoencoder AE-1 is able to provide a dimensionality reduction where the trading activity on
April 17, 2020 is treated as “normal” for a large fraction of investors. Therefore, the
decrease of the overlapping could be ascribed to the change in the distribution of the
times corresponding to the largest anomaly scores.
Among the first 500 ranked investors identified by AE-1 or AE-4, investors who are
detected by AE-1 or AE-4 and that are not detected by the method based on k-means
are analogous to the ones which were identified by relying on PCA and not by the
method based on k-means i.e. profiles like the one in the top left panel of Figure 4,
with one buying transaction just before the beginning of the investigation period.
Instead, among the first 500 ranked investors identified by AE-1, 62 are not identified by PCA. Among them, the investors ranked 15 (Figure 15), 114, 146, 396 are
identified as hard by the method based on k-means. The others are of the type represented in the top left panel of Figure 4. In a similar way, among the first 500 ranked
identified by AE-4, investors who are not detected by PCA are 16. Among them, 3 are
hard and are like the investor in Figure 17, the others are like the one in the top left
panel of Figure 4. It is evident the ability of the autoencoders to capture as anomalous
a type of profile like the ones in Figure 15 – 17, that were not identified by PCA and
that are hard according to the method based on k-means. Moreover, again, the method
based on dimensionality reduction approaches shows its independence of the choice of
the investigation period.
If we compare the results obtained by employing different architectures of autoencoders, among the first 500 ranked by AE-4, only 1 was not detected by AE-1.
Other case studies
While we have extensively covered the case study related to the asset IMA, we now focus
on the other PSEs shown in Table 1. Table 5 summarizes the main results obtained
Asset
PANARIAGROUP
CARRARO
MOLMED
13,225
31,970
1,068
4,500
11,976
1,246
1,801
I/|AKM |
805/857
1,255/1,432
178/188
431/500
465/1,264
Table 5: Anomaly detection on all assets, obtained by employing PCA. N and T are the
numbers of investors and days in the data set. K is the encoding dimension. A is the set of
investors identified as anomalous. I is defined as I = |A?AKM | where AKM is set of investors
identified as hard/soft by the method based on k-means. In is defined as In = |An ? AKM |
where An is the set of the first n ranked anomalous investors.
Model
||X ? X||
205.8
234.0
185.6
168.3
97.07
96.20
97.61
98.04
s¯?
s¯? anomalous
0.4953
0.5982
0.5225
0.6123
0.4417
0.5267
0.4003
0.5015
s¯? normal
0.4905
0.5183
0.4377
0.3955
0.2196
0.1816
0.2034
0.2684
1.208
1.172
1.192
1.253
Table 6:
reduction. EVS is the explained variance score and it is defined
UBI: dimensionality
V ar(X?X)
as 100 1 ? V ar(X) ; s is the mean anomaly score; s¯? anomalous is the mean anomaly score
for investors who are identified as hard/soft by the method based on k-means; s¯? normal is the
mean anomaly score for all investors except the ones identified as hard/soft by the method
¯?
¯?
?s¯? normal
based on k-means; M1 = s anomalous
. The results related to the
; M2 = s anomalous
s¯? normal
s¯?
autoencoders are averaged over 10 runs.
Method
I/|AKM |
1,801 1,255/1,432 10
2,106 1,348/1,432 10
Table 7: UBI: anomaly detection. The encoding dimension is 16. A is the set of investors
identified as anomalous. I is defined as I = |A ? AKM | where AKM is set of investors
identified as hard/soft by the method based on k-means. In is defined as In = |An ? AKM |
where An is the set of the first n ranked anomalous investors.
by using PCA. The overlapping with the results of the method based on k-means are
analogous to what is obtained for IMA, except for MOLMED. For this asset, the small
overlapping is due to the choice of the investigation period, thus highlighting the ability
of our method based on a dimensionality reduction approach to be independent of the
choice of the investigation period. Moreover, the value added by our new method to
the insider trading detection task, is analogous to what is obtained for IMA.
Now, let us deepen the main results related to the asset UBI. We employ both
PCA and autoencoders. In Table 6, a comparison between the reconstruction results
obtained by employing different architectures is shown. A trend different from IMA
can be observed. In this case, we need at least 3 hidden layers to outperform PCA
in the reconstruction of trading profiles and AE-3 shows the lowest MSE. Contrary to
IMA, the architecture which leads to the greatest values of M1 and M2 is still AE-3. If
we rely on this autoencoder and apply our anomaly detection method, the results we
obtain are provided in Table 7, compared with the ones of PCA.
If the first 500 ranked by PCA are considered, our new method does not provide
new information compared to the one based on k-means. This is in contrast with the
autoencoder. The profiles detected by AE-3 and not by k-means are analogous to the
one in the top left panel of Figure 4. On the other hand, the investors detected by AE-3
and not by PCA are hard according to the method based on k-means and analogous to
the profile in Figure 17. The ability of autoencoders to capture this type of investors
which are not detected by PCA, was already shown in the study related to IMA.
Conclusion
We proposed a novel unsupervised approach for contextual anomaly detection, to support decision in insider trading detection. This method tackles the same issue of our
previous paper [19] with a different point of view. In particular, the method based
on k-means, that we develop in [19], is based on the definition of three features i.e.
signed turnover, magnitudo, maximum exposure. With this new method, we aim at
overcoming the features’ choice: our only input is the trading position of each investor
for a given asset and the model learns the relevant characteristics by itself.
This new approach lies in the reconstruction-based paradigm of anomaly detection
and it involves several steps. First, we employ PCA or autoencoders and we obtain the
reconstruction errors for the trading profiles of each investor active on a given asset for
which we have a takeover bid. Then, we localize the largest errors and impose several
conditions in order to detect anomalous investors, who could be suspicious of insider
trading related to the PSE.
We observe a consistent overlapping with the results of the method based on kmeans. However, the value added of this new method is evident. If PCA is employed
as dimensionality reduction approach, the method is extremely fast and easy to implement. Both with PCA and autoencoders, we do not longer have to choose the trading
features which allow to characterize the trading activity of each investor. The method
is not strictly dependent on the choice of the beginning of the investigation period and
actually, it could provide insight on whether this time window should be fixed. The
method is also independent of the choice of the initial time for the computation of the
trading positions.
The differences between the performances of PCA and autoencoders are case-bycase dependent. We showed that autoencoders allow to identify as anomalous, profiles
that are not detected by PCA and are actually hard according to the method based
on k-means. We think that for small data sets, PCA is a sufficient method to perform the dimensionality reduction step. Instead, for larger data sets, a coupled use of
PCA and autoencoders should be preferred. This conclusion is also motivated by the
extreme complexity of our problem, that is also strengthened by the unavailability of
labels, which force us to evaluate the performance of our method without a systematic
procedure.
A natural extension of this work is the employment of more complex architectures
of autoencoders.
List of abbreviations
AE: Autoencoder
CARRARO: Carraro S.p.A.
EVS: Explained Variance Score
IMA: Industria Macchine Automatiche S.p.A.
LAE: Linear Autoencoder
LSTM: Long Short-Term Memory
MOLMED: MolMed S.p.A.
MSE: Mean Squared Error
PANARIAGROUP: Panariagroup Industria Ceramiche S.p.A.
PCA: Principal Component Analysis
PSE: Price Sensitive Event
ReLU: Rectified Linear Unit
SVD: Singular Value Decomposition
UBI: UBI Banca – Unione di Banche Italiane S.p.A.
Declarations
Availability of data and materials
The data that support the findings of this study are from the Commissione Nazionale
per le Societ`
a e la Borsa (CONSOB), that is the public authority responsible for regulating the Italian financial markets. Restrictions apply to the availability of these