Let $Y$ be a space, and $G$ a group. For simplicity we can take $G$ to be finite, $Y$ to be a point, if this avoids technical issues. Then for $X/Y$ a $G$ torsor, so a sheaf of sets with $G$ action on $X$, locally isomorphic to the sections of $G\times U \xrightarrow{\pi} U$, we can consider the automorphisms of this map, in the category of $G$ torsors.


This will be a sheaf of groups on $Y$, and if we choose a section over $U$, then we get base points of all of our fibres, so we get a local isomorphism with the constant sheaf of groups with value $G$. Then changing our section changes our isomorphism, by conjugation.


My question is then, given a sheaf of groups $G$, how can we tell whether it comes from a $G$ torsor in this manner? Or, what gadget parametrises families of these objects, where we know these groups are locally isomorphic to $G$, only canonically up to conjugation. For instance, if $G=A$ is abelian, then this automorphism sheaf will always be a constant sheaf with group $A$.



What extra structure does the group of automorphisms of a torsor carry?
I am not sure what kind of criterion you are after. You can give an answer in terms of cocycles or an answer in terms of killing an obstruction. Briefly, if $\mathcal{G}$ is a bundle of groups on $Y$ which is locally isomorphic to the trivial bundle of groups $\underline{G}$ with fiber $G$, then the first obstruction to $\mathcal{G}$ being the bundle of automorphisms of a $G$ torsor is that $\mathcal{G}$ is an inner twist of $\underline{G}$. Concretely this means the following. Consider the frame bundle of
$\mathcal{G}$, i.e. the bundle of fiberwise group isomorphisms
$Isom(\mathcal{G},\underline{G})$. This frame bundle is manifestly a right $Aut(G)$-torsor (via postcomposition) and the condition is that the structure group of $Isom(\mathcal{G},\underline{G})$ reduces from $Aut(G)$ to the subgroup $Inn(G) \subset Aut(G)$ of inner automorphisms of $G$. Concretely this means that if you look at the assoiciated $Out(G)$-torsor $Isom(\mathcal{G},\underline{G})/Inn(G)$, then this $Out(G)$-torsor is trivializable. Choosing a trivialization gives you an $Inn(G)$ torsor $P$, such that $Isom(\mathcal{G},\underline{G})$ is associated with $P$ via the injective homomorphism $Inn(G) \to Aut(G)$. Once this necessary condition is satisfied and $P$ is chosen ($P$ is well defined up to isomorphism), then you have a second condition which says that the $Inn(G)$-torsor $P$ lifts to a $G$-torsor $X$. From the short exact sequence
$$
1 \to Z(G) \to G \to Inn(G) \to 1,
$$
you see that the obstruction to the existence of such a lift is a degree two cohomology class $\alpha(P) \in H^{2}(Y,Z(G))$ with values in the center $Z(G)$ of $G$. The class $\alpha(P)$ is sometimes called the Brauer class of $P$ and if you choose e.g. a fine enough trivializing cover of $P$ and bounding cochain for the Cech cocycle corresponding to $\alpha(P)$ you can construct a $G$-torsor lifting $P$ explicitly.


