The interpretation of quantum mechanics is controversial because it is impossible to identify individual measurement outcomes with an underlying measurement independent reality [1,2]. Unfortunately, the familiar terminology of quantum mechanics tends to encourage images of underlying realities by carelessly identifying quantum state components with measurement outcomes even when no such measurements are ever performed. It cannot be stressed enough that quantum mechanics contradicts such fantasies directly by explaining the relation between different measurement contexts in terms of superpositions of their outcomes. Still, quantum mechanics does not seem to provide any good alternative. How can it be that we can describe the failure of objective realism in great detail, but cannot find any good alternative that would explain the relation beteen different measurement contexts in a manner consistent with the highly successful theoretical formalism? Perhaps we need a more appropriate formulation of the problem, so that we can identify the correct relations between different measurements without any speculation about unseen realities.

A very good illustration of the problem was recently given by Frauchiger and Renner, who pointed out that the paradoxical correlations between quantum systems observed in a scenario similar to Hardy’s version of a Bell non-locality prevent any consistent description of measurement processes within a quantum system [3,4]. The measurement problem can thus be identified with the problem of contextuality originally explored by Kochen and Specker [5]. To solve the measurement problem, we need to understand how the quantum formalism relates different contexts to each other. Fortunately, we can build on a vast literature concerning this problem. Here, we focus on a three dimensional Hilbert space, the most simple system in which Kochen-Specker contextuality can be demonstrated [6,7,8,9]. The demonstration of contextuality requires a set of five different measurement contexts related to each other through shared measurement outcomes. The quantum formalism then predicts the violation of an inequality that would apply to all non-contextual theories of measurement. It is thus easy to prove that quantum theory must be contextual. However, surprisingly little effort has been made to relate the assumptions of non-contextual theories directly to the quantum formalism. Is it possible to formulate quantum theory in such a way that the difference between its predictions and non-contextual theories becomes more apparent?

Conventional formulations of quantum mechanics tend to prefer a specific context in the formulation of quantum states. To better understand the origin of contextuality in the quantum formalism, we need to investigate statistical expressions that relate different measurement contexts to each other. One such approach is to interpret the quantum formalism as a generalized probability theory [10,11,12,13]. Not surprisingly, the results show that the quantum formalism introduces non-positive quasi probabilities into the statistics, which explains the differences between the predictions of quantum mechanics and the non-contextual hidden variable theories proposed as an alternative. The problem is that there is a wide range of possible quasi probabilities based on different selections of measurements or different operator expansions [14]. It may therefore be more promising to investigate the relation between quantum states and experimentally observable statistics in more detail. Here, I will take a look at the possible reconstruction of quantum states from experimentally accessible data, also known as quantum state tomography. Originally, these methods emphasized the similarity between quantum theory and classical statistics, e.g. when a semi classical analysis is used to reconstruct a phase space representation of the quantum state [15]. It has also been shown that weak measurements or a variety of joint measurements can be used to reconstruct a Kirkwood-Dirac distribution of two non-commuting observables [16,17,18,19,20,21]. These approaches to quantum tomography show that the reconstruction of an unknown state can be achieved by applying methods of classical statistics to characterize the non-classical correlations between the outcomes of incompatible measurements. Non-positive quasi probabilities are obtained because these correlations cannot be explained by a positive valued probability distribution over the possible combination of incompatible outcomes. As observed very early in the history of quantum mechanics, any attempt to describe quantum states as joint probabilities of complementary observables necessarily results in such non-positive distributions [22,23].

In the following, I introduce a method of tomography that relates five different contexts to each other, highlighting the statistical features responsible for the demonstration of quantum contextuality in a three level system [24]. The contexts involved in this tomographical reconstruction can be visualized using a recently introduced three-path interferometer [25]. Non-contextual realities correspond to hypothetical paths of a classical particle through the interferometer. The tomographic method provides the operator algebra needed to identify the different paths with elements of the context independent quantum statistics. Relations between different measurement contexts can then be identified with specific elements of the operator algebra. The results of the analysis show that the quantum state can be reconstructed from a set of only five Kirkwood-Dirac quasi probabilities. Each of these Kirkwood-Dirac terms describes the statistical weight of one of the eleven quasi realities describing the path of a classical particle. Operator algebra can be used to derive five relations that define the Kirkwood-Dirac terms of the remaining paths. These five relations between the different contexts express the deterministic structure of the Hilbert space formalism, preventing an arbitrary assignment of hidden realities even in cases where the individual measurement results would allow it. The framework established by the tomographic approach to quantum contextuality thus provides details of quantum contextuality that are hidden by the “black box” approach of no-go theorems.

Quantum theory makes precise statements about the relations between different contexts. These statements show that quantum mechanics is not constraint by the assumptions of a measurement independent reality. Instead, the uncertainties of the initial state obtain a different meaning in each of the different contexts. This is not just an interpretation of the formalism, the universality of tomographic relations indicate that these are fundamental laws of physics. By designing experiments with sufficient care, it is possible to characterize the specific dependence of reality on the type of meaurement that will eventually be performed. The relations presented in the following are a first step towards a fully deterministic description of the physical world that resolves the measurement problem by identifying the precise role that the measurement context plays in shaping the observable reality of all objects. It should be noted that this kind of progress is made possible by the realization that there is no need for a measurement independent reality. The application of the formalism strongly suggests that the familiar structure of reality that we experience in our actual lives is sufficiently explained in terms of the deterministic relations between different measurements. This is a point that is often overlooked in discussions of the measurement problem such as [1,2]. We do not have any direct access to any microscopic realities, and our ability to perform measurements is limited by our macroscopic interventions in the world around us. If we cannot reconcile our actual experience with our descriptions of “reality,” we need to question the description and not the experience.

Many fundamental investigations of quantum statistics start with the two dimensional Hilbert space of a logical qubit. The physical analogy is a spin-1/2 system, where every direction in space is a potential measurement basis. It is difficult to formulate a logical relation between the different measurement contexts, since no two measurements can be performed jointly and each measurement outcome belongs to only one possible measurement context. The situation changes when a three dimensional Hilbert space is considered. It is now possible to construct a much wider range of possible measurements, and two different measurement contexts can share the same measurement outcome. The Kochen-Specker theorem uses such relations to investigate the statistical relations between incompatible measurements, demonstrating that the relation between different measurement contexts cannot be described by uncertainty limited classical statistics [5]. In the most simple scenario, five different measurement contexts are related to each other by shared measurement outcomes in a three dimensional Hilbert space [6,7,8,9]. As shown in Figure 1, the symmetry of this scenario can be illustrated by a pentagram, where each measurement context is represented by a triangle and the shared outcomes form a pentagon at the center. Note that this representation is a more complete version of the graph derived in [24]. I am using the same terminology, with the context $\{1,2,3\}$ at the top, the contexts $\{1,S1,D1\}$ and $\{2,S2,D2\}$ sharing the outcomes 1 and 2 at their respective sides, and the contexts $\{S1,f,P1\}$ and $\{S2,f,P2\}$ at the bottom, joined by the shared outcome f.

The conventional formulation of quantum theory forces us to chose a basis in which to represent the states and the operators. This can cause serious misunderstandings, since each basis only represents one of the five possible measurements. It is important to remember that the physics of a quantum system should be independent of the basis chosen to describe it. Although Figure 1 correctly identifies the shared measurement outcomes, it remains unclear what the relations between the other outcomes are. In fact, quantum theory defines the relation between two measurement contexts in terms of a unitary transformation, and in the case of the five contexts shown in Figure 1, the transformations between adjacent contexts involves a transformation in the two-dimenional subspace orthogonal to the shared context [24]. As I have shown recently, this relation can be illustrated by a three-path interferomer, where the measurement outcomes of the different contexts are represented by the paths along which a single quantum particle propagates through the interferometer [25]. The transformation between adjacent contexts is then implemented by beam splitters that control the overlap between the quantum states representing the paths. Figure 2 shows the interferometer. Each of the five beam splitters can be identified with the path parallel to it, corresponding to the shared measurement outcome of the contexts before and after the beam splitter. As was shown in [24,25], selecting any two beam splitter reflectivities will determine the other three. An option that is particularly easy to analyze is obtained when $R_1=R_2=1/2$, which results in the remaining reflectities being given by $R_{S1}=R_{S2}=1/3$ and $R_f=1/4$.

What can be said about the relation between different measurement contexts? The identification of measurement contexts with the different stages of single-particle propagation in a three-path interferometer illustrates this relation by associating measurement outcomes with physical paths. A non-contextual hidden variable model would then assign a specific path through the interferometer to each particle. Since the input context $\{1,2,3\}$ is perfectly recovered at the output ports, paths originating from input 1 must end up in output 1 and so forth, leaving a total of eleven distinct possibilities for the propagation of the particle. Table 1 illustrates the eleven paths by assigning a sequence of Hilbert space vectors to each of them. As the table shows, two path segments $[a,b]$ are sufficient to define each of the paths, except for the path composed of the five segments that are unique to each context, which is therefore labeled $\left[0\right]$. The quantum mechanical relations that replace the sequence of paths in the Hilbert space formalism are inner products of the state vectors that represent the measurement outcomes of the different contexts [24]. Although quantum contextuality shows that quantum states cannot be represented by positive valued probability distributions over the eleven paths in Table 1, it should be possible to find a quasi-probability representation of the density matrix that relates directly to these 11 paths. It has already been noted elsewhere that Kirkwood-Dirac quasiprobabilities are ideally suited for this purpose [25,26,27]. In the following, I will develop an expansion of the density matrix that can serve as a starting point for a detailed investigation of the relations between the five measurement contexts described by the Hilbert space formalism.

As explained in previous work, Kirkwood-Dirac distributions are the most consistent representation of quantum statistics, accurately reproducing the relations between non-orthogonal states in the Hilbert space formalism [28,29,30,31,32,33,34,35]. Such relations represent the non-classical structure of operator statistics and ensure that quantum theory cannot be reconciled with any classical statistical theory [14,31]. Here, the goal is to identify a specific set of relations between five different measurement contexts that make optimal use of the orthogonality relations used to demonstarte quantum contextuality. To achieve this, we can start with a selection of non-orthogonal states representing one of the five trajectories composed of three states, e.g. $\{\left|1\right.〉,\left|f\right.〉,\left|D2\right.〉\}$. These three state vectors form a complete non-orthogonal basis of the three-dimensional Hilbert space. The corresponding set of contravariant vectors is $\{\left|S2\right.〉,\left|2\right.〉,\left|S1\right.〉\}$, and the expansion of any state vector in this non-orthogonal basis can be expressed by representing the identity operator as a sum of three non-hermitian operators,

The decomposition of the identity operator $\widehat{\mathrm{I}}$ into non-hermitian operators provides us with an interesting alternative to the decomposition into projectors of orthogonal states used in conventional expansions of the quantum state. The individual non-hermitian operators in Eq.(1) can be used to represent the pre- and post-selected ensembles of weak measurements [36] and appear in the reconstruction of quantum states using direct measurements of the Kirkwood-Dirac distribution [16,17,29]. When interpreted statistically, these operators thus represent the quantum mechanical analog of a logical AND. To simplify the terminology, we can define the operator

As shown in [36], these operators describe the weak measurement statistics of an initial state $\left|a\right.〉$ under the post-selection condition $\left|b\right.〉$. The expectation values of these operators for the states $\left|a\right.〉$ and $\left|b\right.〉$ are necessarily one. Likewise, they are necessarily zero for every state that is orthogonal to either $\left|a\right.〉$ or $\left|b\right.〉$. These properties explain why the identity operator can be decomposed in the manner shown in Eq.(1). In total, there are five operator relations of this kind,

It should be noted that each combination $(a,b)$ corresponds to one of the paths in Table 1. The three operators in each relation represent paths that do not share the same outcome in any of the five contexts. When the basis states of one of the contexts are used to define the matrix representations of the operators, the diagonal elements will be one for the state that is part of the path, and zero for the two other states. For example, the state $\left|3\right.〉$ is part of paths $[S1,D2]$, $[f,3]$, $[S2,D1]$ and $[S1,S2]$. Therefore,

For all other operators $\widehat{\Lambda }(a,b)$,

The operators $\widehat{\Lambda }(a,b)$ thus provide a quantum mechanical representation of 10 of the 11 paths shown in Table 1. In the following, these operators and the related expansion of arbitrary quantum states will be used to derive a method of quantum tomography that highlights the symmetry between the five different contexts using a multi-context version of a Kirkwood-Dirac quasi probability.

The completeness relation in Eq.(1) can be used to express any pure state $\left|\psi \right.〉$ as a superposition of the three states $\{\left|S2\right.〉,\left|2\right.〉,\left|S1\right.〉\}$, where the coefficients are given by inner products with the states $\{\left|1\right.〉,\left|f\right.〉,\left|D2\right.〉\}$,

It should be noted that non-orthogonal expansions reveal a fundamental problem with the notion that each of the components represents a possible “reality”. The coefficients of the expansion are not zero when the state is orthogonal to that component, but instead refer to a different set of states. This means that expansions of states orthogonal to one or two of the components will include non-zero contributions of those components. The problem is best illustrated by the expansion of the state $\left|D2\right.〉$,

Destructive interference between $\left|2\right.〉$ and $\left|S1\right.〉$ ensures that $\langle S2|D2\rangle =0$, and destructive interference between $\left|S2\right.〉$ and $\left|S1\right.〉$ ensures that $\langle 2|D2\rangle =0$. This requires non-trivial relations between the inner products,

As this example shows, all of the inner products are related to each other by the orthogonality relations between the five different measurement contexts [24]. To simplify the relations, it is useful to introduce a specific numerical example. As noted in [24], the example can be defined by the expansion of the states $\left|S1\right.〉$ and $\left|S2\right.〉$ in the $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ basis. It is therefore convenient to chose equal superpositions,

All other superpositions can then be derived using the orthogonality conditions of the relations between the five measurement contexts. Most notably, $\left|f\right.〉$ is orthogonal to both $\left|S1\right.〉$ and $\left|S2\right.〉$, so the representation of the state in the $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ basis is necessarily given by

With these definitions, the expansion of $\left|D2\right.〉$ in Eq.(7) is given by

Note that the physics represented by the state $\left|D2\right.〉$ cannot depend on its representation. The characterization of an unknown state should not be based on the components of a representation, but on the coefficients of the expansion and any physical meaning that they may have that is independent of the expansion in which they are used.

The coefficients of the non-orthogonal expansion of quantum states given by Eq.(6) describe coherences between non-orthogonal states. Such coherences can be represented by Kirkwood-Dirac quasi probabilities,

As mentioned before, the Kirkwood-Dirac quasi probabilities describe the relation between the quantum statistics of $\left|a\right.〉$ and $\left|b\right.〉$ in terms of the weak values of $\left|a\right.〉\left.〈a\right|$ post-selected in $\left|b\right.〉$ [16,17]. In the following, I will refer to them as KD terms. When the non-orthogonal expansion of Eq.(6) is applied to a density operator $\widehat{\rho }$, the coefficients of the expansion can be expressed by a combination of probabilities and KD terms,

The expansion combines the probabilities of the three outcomes 1, f and $D2$ with KD terms for all three combinations, $(1,f)$, $(1,D2)$ and $(f,D2)$. These six elements correspond to the diagonal and off-diagonal terms of a density matrix, where the coherences are replaced by complex quasi probabilities that assign a joint statistical weight to each pair of outcomes.

The expansion given in Eq.(13) did not require any normalization of the density operator $\widehat{\rho }$. It is therefore possible to derive an additional relation between the six coefficients using the condition $Tr\left(\widehat{\rho }\right)=1$. For example, it is possible to express the probability of $D2$ as

A normalized quantum state is completely determined by the two probabilites $P\left(1\right)$, $P\left(f\right)$, and the three KD terms $\varrho (1,f)$, $\varrho (1,D2)$, $\varrho (D2,f)$. Experimentally, KD terms can be determined by weak measurements or by a variety of other methods involving quantum interferences [16,17,18,19,20,21]. Eqs.(13) and (14) show that the statistics of all possible measurements can be expressed by linear combinations of these five coefficients and the normalization. In the following, we will take a look at how this expansion of the quantum state relates different contexts to each other.

A Kirkwood-Dirac distribution describes the relation between two different measurement contexts in the form of a complex joint probability of all possible combinations of outcomes. If the inner products of outcomes from different contexts are non-zero, the Kirkwood-Dirac distribution of these contexts is a complete representation of the quantum state [16,17,28,29]. However, this procedure cannot be applied to any of the five contexts, since at least two of the outcomes from different contexts will be orthogonal to each other. It is therefore not surprising that the expansion in Eq.(13) combines outcomes from all five contexts. The selection of the specific expansion given by Eq.(1) defines expansion coefficients associated with the path $[1,f]$ in Table 1. It is possible to relate these coefficients to a complete Kirkwood-Dirac distribution for the contexts defined by the unique outcomes $\left|3\right.〉$ and $\left|P2\right.〉$. This Kirkwood-Dirac distribution has a total of eight non-zero elements, as shown in Table 2. Five of these elements are needed to determine the five coefficients of the expansion in Eq.(13). It is conventient to use the five coefficients marked in red, since the coefficients can then be determined by simple sums. The probabilities $P\left(1\right)$ and $P\left(f\right)$ are marginals of the distribution,

The remaining two coefficients are given by KD terms that involve $D2$. Since $\left|S2\right.〉$ is orthogonal to $\left|D2\right.〉$, $\varrho (1,D2)$ does not include the contribution of $\varrho (1,S2)$ to $P\left(1\right)$. The same holds true for $\varrho (D2,f)$ and $\varrho (2,f)$. The two remaining KD terms in the expansion are therefore given by

Only five of the eight elements of the Kirkwood-Dirac distribution for the contexts $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ and $\{\left|f\right.〉,\left|S2\right.〉,\left|P2\right.〉\}$ are needed to completely define an arbitrary quantum state. The relation between the contexts represented by the orthogonality of $\left|2\right.〉$ and $\left|S2\right.〉$ thus simplifies the characterization of an unknown state.

The Kirkwood-Dirac distribution is a helpful representation of the relations between two different contexts. It is therefore interesting to investigate how Eq.(13) relates the five coefficients that appear in the expansion to the remaining three elements of the Kirkwood-Dirac distribution in Table 2. The easiest way to find the result is to consider the probabilities of $S2$ and 2,

These marginals of the Kirkwood-Dirac distribution determine the KD terms $\varrho (3,S2)$ and $\varrho (2,P2)$ from the elements marked in red in Table 2. The remaining element $\varrho (3,P2)$ can be determined from the normalization of the distribution. Since $\left|2\right.〉$, $\left|S2\right.〉$ and $\left|D2\right.〉$ form the orthogonal basis of a third context, it is also possible to determine $\varrho (3,P2)$ from the probability of $D2$ as given in Eq.(14). The three probabilities are related to the elements of the Kirkwood-Dirac distribution by

The remaining elements of the Kirkwood-Dirac distribution can then be expressed by the five KD terms used in the reconstruction of the quantum state,

These equations represent a set of universal rules that relate the elements of a single Kirkwood-Dirac distribution to each other. It is worth looking at these rules in detail, to figure out what kind of statistical relations they express.

The most obvious reason why there should be a relation between the elements is the normalization of the distribution. Probabilities attributed to the five elements used in the reconstruction will be missing from the three additional elements. If all of the elements used in the reconstruction are zero, the state is $\left|S1\right.〉$ and the three additional elements are $\varrho (3,P2)=1/4$, $\varrho (2,P2)=1/2$, and $\varrho (3,S2)=1/4$. It is interesting to observe that there is no degree of freedom left. There is only one state with $P\left(1\right)=P\left(f\right)=0$. Eq.(19) then defines the necessary uncertainty of this state in terms of a joint probability of the remaining outcomes $(2,3)$ and $(S2,P2)$. We can also consider cases where there is only one non-zero element among the five elements used in the reconstruction of the state. Most obviously, this is the case for $\left|2\right.〉$, where $\varrho (2,f)=1/3$, and for $\left|S2\right.〉$, where $\varrho (1,S2)=1/2$. In both cases, a contribution proportional to the non-zero element is subtracted from two of the three elements in Eq.(19), leaving only one additional element with a non-zero value. Naturally, this element obtains exactly the value needed for a total probability of one, $\varrho (2,P2)=2/3$ for $\left|2\right.〉$ and $\varrho (3,S2)=1/2$ for $\left|S2\right.〉$. It is worth noting that a value of $\varrho (2,f)$ greater than $1/3$ would result in a negative value of $P\left(S2\right)$, indicating a strict limit to the concentration of quasi probabilities in a single KD term. A lower value of $\varrho (2,f)$ is possible, since it corresponds to a mixed state of $\left|S1\right.〉$ and $\left|2\right.〉$. The reason for the upper limit of $\varrho (2,f)$ is that the reconstructed state would be a non-positive state - specifically, a mixture of $\left|2\right.〉$ and $\left|S1\right.〉$ with a negative contribution from $\left|S1\right.〉$.

Finally, it may be instructive to consider states with particularly high probabilities of 1 and f, since the reconstruction method and the Kirkwood-Dirac distribution put special emphasis on these two outcomes. For $\left|f\right.〉$, the KD-terms are $\varrho (1,f)=1/3$, $\varrho (2,f)=1/3$ and $\varrho (3,f)=1/3$, while $\varrho (1,S2)=\varrho (1,P2)=0$. As expected, Eq.(19) then states that the remaining three elements of the distribution are zero. Any attempt to re-distribute probabilities between the three non-zero elements will result in a re-distribution of a total probability of zero between $P\left(S2\right)$ and $P\left(P2\right)$, resulting in at least one negative probability. Eq.(19) thus allows only one probability distribution of 1, 2 and 3 for $P\left(f\right)=1$. The same holds true for the state $\left|1\right.〉$, where the only possible probability distribution is $\varrho (1,f)=1/3$, $\varrho (1,S2)=1/2$ and $\varrho (1,P2)=1/6$, while $\varrho (2,f)=\varrho (3,f)=0$.

Since we are interested in the relations between the two contexts in the Kirkwood-Dirac distribution, we can now take a look at a state that achieves high probabilities for both 1 and f simultaneously. Since such a state closely approximates the trajectory $[1,f]$, I will refer to it as the state $\left|T_{1f}\right.〉$. To keep the numerics simple, let us consider the superposition

This state has $P\left(S1\right)=0$, so that it can be represented as a superposition of $\left|1\right.〉$ and $\left|f\right.〉$. The interference between these two components is constructive, adding a probability of $4/11$ to the probabilities associated with the expansion coefficients. The state can also be written as

When written in this form, it is easy to derive the probabilities for any of the five measurement contexts. For $\left|1\right.〉$, we get contributions of 1 from all operators except for $\left|f\right.〉\left.〈f\right|$, where the contribution is $1/3$. The total probability is therefore $P\left(1\right)=9/11$. Similarly, $\left|f\right.〉$ results in contributions of 1 except for $\left|1\right.〉\left.〈1\right|$, where the contribution is 1/3. The total probability is $P\left(f\right)=25/33$. In classical statistics, these probabilities would suggest a joint probability of 1 and f of at least $19/33$, obtained under the assumption that the system is always either in f or in 1. However, the corresponing KD term is only $\varrho (1,f)=15/33$. This is possible because negative values appear in the KD terms that are neither related to 1 nor to f.

Table 3 shows the complete Kirkwood-Dirac distribution of the state $\left|T_{1f}\right.〉$. According to Eq.(19), an increase in $\varrho (1,f)$ will result in a decrease of $\varrho (3,S2)$ and $\varrho (2,P2)$. In the present case, we find that $\varrho (3,S2)=-3/33$ and $\varrho (2,P2)=-2/33$. The third KD term in Eq.(19) has a positive value of $\varrho (3,P2)=1/33$. Eq.(19) shows that these three values correspond an addition of $+4/33$ to $\varrho (1,f)$ compared with a situation where all three KD terms are initially zero, the situation obtained for any incoherent mixture of $\left|1\right.〉$ and $\left|f\right.〉$. The superposition state $\left|T_{1f}\right.〉$ thus corresponds to a mixture of $\left|1\right.〉$ and $\left|f\right.〉$ with an addition of $+4/33$ to $\varrho (1,f)$. Specifically, a mixture of $6/11$ times $\left|1\right.〉\left.〈1\right|$ and $5/11$ times $\left|f\right.〉\left.〈f\right|$ has the same values for $\varrho (1,S2)$, $\varrho (1,P2)$, $\varrho (2,f)$ and $\varrho (3,f)$ as $\left|T_{1f}\right.〉$, but all of the remaining probability are found in $\varrho (1,f)=11/33$. The coherence between $\left|1\right.〉$ and $\left|f\right.〉$ increases the value of $\varrho (1,f)$ by $+4/33$, and this increase in $\varrho (1,f)$ results in increase of $1/4$ times $4/33$ or $1/33$ in $\varrho (3,P2)$ a decrease of $-1/2$ times $4/33$ or $-2/33$ in $\varrho (2,P2)$, and a decrease of $-3/4$ times $4/33$ or $-3/33$ in $\varrho (3,S2)$. Eq.(19) thus shows that negative KD terms are needed to achieve high values in other KD terms. The analysis in this section has shown that these relations are already obtained when only two contexts are related to each other. This is especially interesting because the Kochen-Specker theorem assigns a cyclic and symmetric role to all five contexts, as illustrated in Figure 1. In the following, I will take a look at the inequality violation used to demonstrate contextuality in the present scenario and use it as a starting point for a formulation of cyclic relations between KD terms that correspond to non-contextual hidden variable theories for the five contexts.

Non-contextual hidden variable models assume that the outcomes of all measurements are pre-determined for each individual system. For the three-path interferometer in Figure 2, this means that each particle follows a specific path through the five contexts, corresponding to the paths shown in Table 1. If this is indeed the case, it is possible to identify extremal values of collective properties that define inequalities for the experimentally observable statistics. In the well-studied case of the five contexts defined here, the maximal number of shared outcomes that a path may include is two. This means that two is the maximal value we should obtain if we sum the probabilities of all shared outcomes [8],

It is well known that this inequality can be violated, and various formulations of this paradox have been discussed [8,9]. Here, our goal is to relate the paradox to the KD terms that describe the relations between different measurement contexts.

In the tomographic reconstruction considered above, we have direct access to $P\left(1\right)$ and $P\left(f\right)$, as shown in Eq.(15). $P\left(2\right)$ and $P\left(S2\right)$ are given in Eq.(17). Since only the sum of $P\left(2\right)$ and $P\left(S2\right)$ is needed, it is sufficient to refer to $P\left(D2\right)$ in Eq.(14) to obtain

This leaves only the probability $P\left(S1\right)$. Using Eq.(13)and Eq.(14), this probability can be given as

The sum of all five probabilities is then given by

An inequality violation is observed whenever the KD terms used in the reconstruction of the state satisfy

Despite its negative KD terms, $\left|T_{1f}\right.〉$ does not satisfy this requirement and therefore fails to violate the inequality in Eq.(22). This is not surprising, since the probability of $S1$ in $\left|T_{1f}\right.〉$ is zero, and the remaining probabilities in the sum $\Sigma$ can be divided into pairs from the same context. In general, the inequality given by Eq.(22) cannot be violated when one of the five probabilities is zero. As pointed out in [8], a state that optimally achieves a violation can be obtained by superpositions of all five states representing the outcomes that contribute to the inequality, simultaneously achieving very similar probabilities for all five outcomes. A state that is close to the maximal inequality violation is given by [37]

The tomographic data for this state is given by

The value of the sum $\Sigma$ obtained with this data is $\Sigma =20/9$, violating the inequality by $2/9$. The individual probabilities are $P\left(1\right)=P\left(2\right)=4/9$, $P\left(S1\right)=P\left(S2\right)=1/2$ and $P\left(f\right)=1/3$.

Eq.(26) indicates that the violation of the non-contextual inequality in Eq.(22) can be explained by the negative $KD$ terms of the paths $[3,f]$ and $[1,P2]$. The tomographic data of Eq.(28) thus explains the inequality violation in terms of its negative KD terms. However, Eq.(22) does not require negative KD terms for its violation, and the specific selection of the tomographic data hides the fact that all five contexts play an equivalent role in the violation of the inequality. It is therefore desirable to derive a more complete form of the inequality, where the role of negativity and its relation to generalized probabilities as discussed in [10,11] is more apparent. In the case of the $\left|N_x\right.〉$ state, we would expect negative KD terms for all paths with four segments in Table 1. In addition to the paths $[3,f]$ and $[1,P2]$, these are the paths $[2.P1]$, $[S1,D2]$ and $[S2,D1]$. If we simply complete the Kirkwood-Dirac distribution, we get the terms $\varrho (3,S2)$, $\varrho (2,P2)$ and $\varrho (3,P2)$. According to Table 1, each of these combinations of outcomes occurs in two of the eleven paths. The Hilbert space algebra confirms that the KD terms can be expressed in terms of the corresponding sums,

All of the KD terms defined by pairs of outcomes can be determined using Eq.(12). The KD term $\varrho \left(0\right)$ of the path through all five unique outcomes needs a different definition. The general definition can be obtained by referring to the normalization of the Kirkwood-Dirac distribution,

This equation can be simplified by summarizing the KD terms associated with 1 and 2, for

Independently, the fact that $\left|S1\right.〉\left.〈S1\right|+\left|D1\right.〉\left.〈D1\right|$ projects states into the Hilbert space spanned by $\left|2\right.〉$ and $\left|3\right.〉$, while $\left|S2\right.〉\left.〈S2\right|+\left|D2\right.〉\left.〈D2\right|$ does the same for $\left|1\right.〉$ and $\left|3\right.〉$ can be used to derive a very similar expression for $P\left(3\right)$,

Based on these relations, a conveniently symmetric definition of $\varrho \left(0\right)$ is

In general, $\varrho \left(0\right)$ contributes to all KD terms defined by pairs of outcomes that are unique to two different contexts.

Eq.(30) shows that all quantum states can be characterized by a quasi probability distribution given by eleven KD terms that correspond to the 11 possible paths through the five contexts shown in Table 1. If all KD terms are positive, these eleven terms can be interpreted as a probability distribution of the possible joint assignment of measurement outcomes to all five contexts. The eleven KD terms can thus establish a link between quantum statistics and hidden variable theories. However, quantum theory introduces additional relations between the eleven KD terms. Since only five KD terms are needed to characterize the quantum state completely, there should be a set of six relations that determine the remaining six KD terms as a function of these five. One of these relations is the normalization, and this relation can be used to determine $\varrho \left(0\right)$ once the other five KD terms have been determined. The remaining five relations can be derived from Eq.(3), where each KD term is given by the expectation value of the operator $\widehat{\Lambda }(a,b)$ according to

Eq.(3) thus defines a set of deterministic relations between the KD terms that express a distribution of statistical weights over the possible quasi realities obtained by assigning a specific measurement outcome to each context. These fundamental constraints satisfied by the KD terms of all possible input states are given by

The KD terms in each line correspond to paths that do not share the same measurement outcome in any of the five contexts. It is therefore possible to assign exactly one KD term to each outcome in each of the five lines. For outcomes that are shared by two contexts, this results in three non-zero KD terms, e.g. $\varrho (1,S2)=1/2$, $\varrho (1,f)=1/3$ and $\varrho (1,P2)=1/6$ for $\left|1\right.〉$. Since the sum is one, the eleventh KD term is $\varrho \left(0\right)=0$. On the other hand, outcomes that are unique to only one context have a total of five non-zero KD terms. For the state $\left|3\right.〉$, these are the four positive KD terms $\varrho (D2,S1)=1/4$, $\varrho (3,f)=1/3$, $\varrho (D1,S2)=1/4$, $\varrho (S1,S2)=1/4$, and the fifth negative KD term $\varrho \left(0\right)=-1/12$.

The appearance of a negative KD term in states associated with a specific measurement outcome may seem surprising, since such a state is related to the other contexts directly by its inner products with the outcomes of those contexts. Indeed, $\varrho \left(0\right)$ is somewhat special, since it cannot be derived from a single weak value and does not appear in any Kirkwood-Dirac distribution of only two measurement contexts. It can be confirmed that, for all outcomes that are unique to only one context, the value of this KD term is $\varrho \left(0\right)=-1/12$. The origin of this value is found in a Bargmann invariant defined by the Hilbert space vectors representing the five outcomes that are unique to each context,

In principle, it is possible to show that Eq.(33) identifies this Bargmann invariant with $\varrho \left(0\right)$ for the states $\left|3\right.〉$, $\left|D1\right.〉$, $\left|P1\right.〉$, $\left|P2\right.〉$ and $\left|D2\right.〉$. However, it might be more useful to show that the value of $\varrho \left(0\right)$ must be the same for each of these states. This can be done by determining $\varrho \left(0\right)$ for a maximally mixed state, $\widehat{\rho }=\widehat{\mathrm{I}}/3$. For that state, $\varrho (a,b)={\left|\langle a|b\rangle \right|}^2/3$ and $\varrho \left(0\right)=-1/36$. Specifically, $\varrho \left(0\right)$ is negative because ${\left|\langle 3|f\rangle \right|}^2>{\left|\langle D1|D2\rangle \right|}^2$. Since $\varrho \left(0\right)$ is zero for each state shared by two contexts, the negative value of the maximally mixed state must correspond to one third of the value of the state unique to that context for each decomposition of the mixed state into a mixture of states representing the measurement outcomes of any of the three contexts.

In general, all quantum states can be described by the 11 quasi probabilities of the different paths through the contexts. However, even mixed states tend to have negative KD values. The only states that result in positive non-contextual probabilities are states that can be expressed as mixtures of the five outcomes that are shared by two contexts. Significantly, this does not include the maximally mixed state $\widehat{\mathrm{I}}/3$ with its negative KD term of $\varrho \left(0\right)=-1/36$. The violation of non-contextuality as given by Eq.(22) requires a very specific combination of negative KD terms. To find these, we can make use of the correspondence between KD terms and classical paths through the contexts. Path $\left[0\right]$ has no shared outcomes, paths $[1,P2]$, $[S1,D2]$, $[f,3]$, $[S2,D1]$, $[2,P1]$ have one shared outcome each, and the remaining five paths have two shared outcomes. The average number of shared outcomes can then be found by subtracting the KD terms of the paths with less than two shared outcomes from an initial value of two,

This expression for the probability sum $\Sigma$ shows that a violation of inequality (22) is impossible unless some of the KD terms given are negative. As suggested by Spekkens in [10], the origin of contextuality paradoxes can be traced to negative KD terms that suppress the contributions of paths with one shared outcome or none at all. Importantly, Kochen-Spekker scenarios require a joint probability characterizing more than two contexts. The eleven KD terms introduced here achieve this for all five contexts, even though each KD term is defined by only two outcomes. This is the significance of Table 1. Quantum contextuality actually provides an extended probability theory that corresponds to the assignment of outcomes to all five contexts at the same time.

The eleven KD terms represent a distribution of joint statistical weights for all five contexts, where each KD term represents a specific combination of outcomes for each of the five contexts, corresponding to one of the paths in Table 1. Like other quasi probabilities, this multi-context Kirkwood-Dirac distribution seeks to bridge the gap between a naive realism that assumes a co-existence of the measurement outcomes along a path and an equally naive anti-realism that simply assumes that there is no connection between the outcomes of different measurements. To guard ourselves against both misconceptions, we should accept that the 11 element Kirkwood-Dirac distribution describes a deterministic relation between the five different measurement contexts that cannot be reconciled with any joint reality of outcomes. To understand the concept of quantum contextuality better, we need to focus on the fact that only one measurement context describes the reality of an actual event. Measurement probabilities are obtained from sums of KD terms for this event, and all of these sums will necessarily be positive. Negative KD terms can never be observed in isolation, since their experimentally observable effects appear only in sums where the contributions from positive KD terms outweigh those from the negative ones. Quantum contextuality suggests that the reality observed in one measurement context is fundamentally different from the reality observed in a different measurement context. However, the KD terms $\varrho (a,b)$ still describe a well-defined relation between the outcomes a and b. We should consider this in more detail - if the relation between a and b is valid when we measure a, then the measurement of a determines the reality of b as well. The set of values provided by the KD terms thus describes a contextual reality of b conditioned by the directly observed outcome a.

The problem that prevents us from understanding quantum mechanics is that we identify physical properties with their eigenvalue, clinging to the believe that these eigenvalues are a complete and exclusive set of possibilities from which we can construct an objective reality. However, this is a very unlikely interpretation of the formalism. Instead, we should consider a much wider variety of possibilities, where the outcomes of the context a are related to contextual realities of outcomes b that are different from the limited options of the binary yes or no associated with the eigenvalues of the projectors [38,39,40,41]. We need to accept that the values of projectors associated with other measurement outcomes can be different from 0 or 1. Is it possible to make intuitive sense of such a possibility? Perhaps a change of terminology is needed - “truth value” applies only to actual observations. Here, I will use the term “outcome value,” or more specifically, “contextual outcome value”. It is easiest to visualize this change by considering the case of a single particle propagating along paths. As has already been demonstrated experimentally [39], a particle is genuinely de-localized between different paths when these paths interfere in the output. We can now generalize this result to all quantum systems by describing the conditional reality of outcomes b as a physical distribution of the total outcome value of 1 over the outcomes b.

The concept of a context dependent reality can be illustrated using the relations between measurement contexts described by the KD terms. Each measurement context has three outcomes, two of which are shared with other contexts and one of which is unique to that context. The measurement probabilities are given by sums of three KD terms for each of the shared outcomes, and by a sum of the remaining five KD terms for the outcome unique to this context. For instance, the probabilities of the context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ are given by

Each measurement outcome assigns a truth value of 1 to one of the three outcomes, and a truth value of 0 to the others. These truth values correspond to eigenvalues of the projection operators $\left|a\right.〉\left.〈a\right|$ of the measurement context. Quantum mechanics strongly suggests that we should not assign such eigenvalues to unobserved measurement outcomes. Instead, weak values can be used to relate the outcomes b to a post-selected outcome of the measurement context a [25,38,39]. Using the terminology above, these weak values are the contextual outcome values of b in the context of a. If the measurement outcome a is obtained with a probability of $P\left(a\right)$, the contextual outcome value $W\left(b\right|a)$ of an outcome b is given by

Technically, $W\left(b\right|a)$ corresponds to the complex conditional probabilities discussed in some of my previous work [28,29], where the status of the Kirkwood-Dirac distribution as a symmetric quasi probability of two measurement contexts was emphasized. However, contextuality suggests that we need to abandon the idea of a truth value of zero or one for outcomes that are not observed in their own context - or “eigencontext” as I shall refer to in the following.

If the contextual outcome values $W\left(b\right|a)$ are considered as individual values of the observable represented by the projector $\left|b\right.〉\left.〈b\right|$, it is possible to define the fluctuations of this value in the context of a by assigning the probabilities $P\left(a\right)$ to the corresponding outcome values $W\left(b\right|a)$. The dependence of $W\left(b\right)$ on a thus represents the fluctuations of b in the context a. Since the average value of $\left|b\right.〉\left.〈b\right|$ is equal to the probability $P\left(b\right)$, the fluctuation of $W\left(b\right|a)$ is given by

As shown in a number of previous works, the fluctuations of weak values over a complete set of post-selected outcomes are equal to the uncertainty of the corresponding observable for pure state inputs [38,42,43]. For the contextual values $W\left(b\right|a)$,

In the limit of pure states and precise measurements, the contextual fluctuations of the outcome values are equal to the fluctuations of the truth value observed in a direct measurement of the corresponding context. This is strong evidence that the values $W\left(b\right|a)$ describe a context dependent reality.

To connect this analysis with quantum tomography, it is necessary to consider the role of the initial state in the definition of contextual fluctuations. If the state is a pure state, it is characterized by a deterministic statement corresponding to a certain prediction of a measurement outcome. Whether this prediction is explicit of not, it has an effect on the outcome values. An example may help to explain this dependence. Let us consider the state $\left|P2\right.〉$. Its eleven KD terms are given by

As explained above, the KD term $\varrho \left(0\right)$ of this state is negative. We can therefore expect to find negative outcome values for some of the outcomes when the measurement context is not the eigencontext of that outcome. For instance, the values $W\left(D1\right|P2)$ of the outcome $\left|D1\right.〉$ and their probabilities in the context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ are given by

Note that I have included the initial state in the conditions that determine the value $W\left(D1\right)$. This is intended as a reminder of the physical meaning of these values. The combination of state preparation $P2$ and measurement outcome m completely describes the physical reality of the system. The value $W\left(D1\right)$ for this physical reality is determined by these conditions, leaving no uncertainty in the value of the projector $\left|D1\right.〉\left.〈D1\right|$. We can confirm this assumption by finding the fluctuation of $W\left(D1\right)$. In every measurement context, the average value of $W\left(D1\right)$ in the state $\left|P2\right.〉$ is equal to the probability of $D1$, $P\left(D1\right|P2)=1/12$. The fluctuations of $W\left(D1\right)$ in $\left|P2\right.〉$ are given by

The fluctuations obtained for the context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ are the same as the fluctuations obtained in the eigencontext of $\left|D1\right.〉$, even though the outcome values themselves are very different. This is a strong indication that the individual realities associated with this fluctuation depend on the measurement context. In its eigencontext, the outcome $D1$ simply has a low probability. In the context $\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$, the main contribution to the expectation value of $P\left(D1\right)=1/12$ originates from the most likely outcome of 2, for which the outcome value of $D1$ is $1/4$, much smaller than a truth value of 1, but still three times larger than the average. The negative outcome value of $-1/2$ for the less likely outcome of 3 is needed to achieve the same average and the same fluctuations as the eigencontext of $D1$. This is similar to the negative presence of a particle in a two-path interferometer that was reported in the neutron interference experiment in [39]. In general, negative contextual outcome values are observed when the probability of the observed outcome is low because of destructive interference between outcomes of the unobserved context. The quantum formalism assigns precise conditional values to these unobserved outcomes, providing a context dependent description of the quantum fluctuations of $D1$ in the input state $\left|P2\right.〉$. Weak values and KD terms represent deterministic relations that allow statements about the values of unobserved physical properties. In the case of projectors, negative values indicate a particularly strong deterministic suppression of contributions from this outcome.

The initial state defines averages and fluctuations. It does not define outcome values and probabilities. These can only be determined when the context is fixed. It is therefore interesting to take a look at states that seek to minimize the uncertainties if outcomes in all five contexts. As discussed above, the state $\left|T_{1f}\right.〉$ achieves this by enhancing the KD term that corresponds to the path $[1,f]$ through the five contexts. Table 3 already shows five of the eleven KD terms for this state. The complete set of eleven KD terms can be obtained from the table by making use of the observation that the $\left|T_{1f}\right.〉$ state has $P\left(S1\right)=0$. Therefore, each of the eight elements in the table appear as one of the eleven KD terms, while the remaing three KD terms have values of zero. The complete set of KD terms is given by

The state $\left|T_{1f}\right.〉$ satisfies the conditions given by Eq. (35) by compensating the enhancement of $\varrho (1,f)$ above its eigenstate value of $1/3$ with two negative KD terms, $\varrho (2,P1)$ and $\varrho (D1,S2)$. Note that the paths $[2,P1]$ and $[S2,D1]$ represented by these two KD terms do not share any outcome with $[1,f]$ in Table 1. This means that simultaneous control of five contexts is achieved at least in part by suppressing joint contributions of the unintended outcomes. We expect to see this effect more clearly in the relation between observed outcomes and contextual outcome values.

We can now consider the statistics of outcome values for f, $S2$ and $P2$ in the measurement context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$. The probabilities of each outcome and the contextual distribution of the total outcome value of 1 over f, $S2$ and $P2$ are given by

The first thing that should be noted is the way in which the high average of $P\left(f\right)=25/33$ for the outcome f is obtained. Instead of assigning a high value to the most likely outcome of 1, the Hilbert space formalism determines a value of only $5/9$ ($11/15$ of the average of $25/33$) to this outcome. Oppositely, the less likely outcomes 2 and 3 have a contextual value of $5/3$ for f, exceeding the eigenvalue limit of 1 of the projector. This is a direct consequence of the relations in Eq.(35), where the high value of $P\left(f\right)$ is achieved by suppressing joint contributions of $(2,3)$ and $(S2,P2)$. This suppression is directly visible in the negative outcome values of $W\left(P2\right)$ for outcome 2, and for $W\left(S2\right)$ for outcome 3. If the distributions of outcome values are viewed together, the concentration of outcome values on f can be observed for each of the outcomes, but it is much weaker for the most likely outcome of 1 and extremely enhanced by outcome values outside the eigenvalue range for the less likely outcomes of 2 and 3.

As noted above, the statistical fluctuations of the outcome values are all equal to the corresponding truth value fluctuations of the eigencontext statistics given by the probabilities of $P\left(f\right)=25/33$, $P\left(S2\right)=2/11$ and $P\left(P2\right)=2/33$. This can be confirmed by considering the differences between the outcome values and the average values represented by the probabilities,

The statistical fluctuations of the outocomes f, $S2$ and $P2$ observed in their eigencontext are fully reproduced by the statistics of the outcome values associated with the outcomes 1, 2 and 3. No additional randomness is required - the fluctuation of the outcomes in the context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ has the same physical origin as the fluctuation of the outcomes in the context $\{\left|f\right.〉,\left|S2\right.〉,\left|P2\right.〉\}$.

As shown above, the inequality violations usually associated with quantum contextuality is a natural consequence of the dependence of outcome values on the measurement context, where negative contextual outcome values appear as fluctuations associated with low probability outcomes. Inequality violations can be achieved when a specific set of KD terms is negative, as shown in Eq.(37). In the case of the state $\left|N_x\right.〉$, five of the six $KD$ terms contributing to the inequality violation are negative. The complete set of eleven KD terms can be obtained from the five terms in Eq.(28) and the relations in Eq.(35). The results read

The value of the sum $\Sigma$ is given by

The inequality (22) is violated because the negative KD terms for paths with one shared outcome outweigh twice the positive KD term of the path with no shared outcome. The $\left|N_x\right.〉$ state is nearly symmetric in the five contexts, with maximal suppression of the outcomes unique to each context. For the measurement context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$, the measurement probabilities and the contextual values $W\left(f\right)$, $W\left(S2\right)$ and $W\left(P2\right)$ are given by

The first thing to note regarding the dependence of the contextual values on the measurement outcomes is the dominance of suppression effects over enghancement effects. For an outcome of 1, the contextual values of f and $S2$ are nearly equal, but the contextual value of $P2$ is negative. For an outcome of 3, the value of $S2$ is greater than one, but the value of $P2$ is also much larger than its average of $P\left(P2\right)=1/12$. More impressive is the suppression of the value of f to $-1$, especially since the average value of f is $P\left(f\right)=1/3$. For the outcome 2, the value of $S2$ is the eigenvalue of zero, but it is worth noting that the conditional values of f and $P2$ are the same. Each outcome has high values for two of the three outcomes, even though the statistics of these outcome values fully explain the input state uncertainties, leaving no room for any uncertainties in the conditional values themselves. Negative contextual values describe the deterministic relations between different contexts as extreme anti-correlations, exceeding the limits based on the assumption that the maximal correlation is a joint reality represented by a conditional value of one for a specific outcome and a value of zero for the two others.