For me, a fusion category (over $\mathbb{C}$) is just a tensor $F$ (the associator, with $6$ simple-object labels and $4$ fusion space indices) and a tensor $d$ (the quantum dimensions, with one simple-object label), such that the $F$-tensor fulfils the pentagon equation. (This seems to be the most useful language for physics.)


If furthermore the fusion category is unitary, this means that (one can find a basis for the fusion spaces such that) the $F$-tensor is unitary when interpreted as a linear map between two groups of its indices.


What extra structure/conditions do I have to add in this tensor language to make the (unitary) fusion category pivotal/spherical?


For example, it seems that a unitary fusion category is spherical if the $F$-tensor is symmetric under the full tetrahedral symmetry group (after normalizing with the square roots of the quantum dimensions of four of its simple-object labels). This seems not to be the most general case, however.



What do "pivotal" and "spherical" mean for (unitary) fusion categories on the level of the $F$-symbols?
Any unitary fusion category has a canonical spherical structure. See Example 2.12 of [this paper](https://arxiv.org/pdf/1611.04620.pdf) and the references therein. It also follows from a more general result Prop 8.23 in [this paper](https://arxiv.org/abs/math/0203060). So you don't need to do anything in the unitary case.


For a non-unitary fusion category, I think you're looking for [Bruce Bartlett's explicit description of pivotal structures](https://arxiv.org/pdf/1502.02882.pdf). There he shows that you can think of a pivotal structure as a choice of a number $\gamma\_i$ attached to each simple object such that the condition of Theorem 5.4 holds:
$$\gamma\_j \gamma\_k \mathrm{Id} = \gamma\_i T\_{jk}^i \text{ whenever $X\_i$ is a summand of $X\_j \otimes X\_k$}.$$
Here $T\_{jk}^i$ are the pivotal operators which are defined from the F-matrices by Thm 4.17. (There's a slightly tricky point there which I'm unclear on, which is that the formula in 4.17 only works for a "fair basis" which should translate into some simple condition on how you've gauaged your F-matrix, but it's not obvious to me what that condition is.) The pivotal structure is spherical if all the $\gamma\_i$ are $\pm 1$.


