The objects of the desired category are epimorphisms of sets $E \to B$ (in what follows, the notation $E/B$ will be used instead of the arrow). Is it possible to naturally define morphisms such that:


1. $E/B \cong (E \times A) / (B \times A)$ for each inhabited $A$ and there are no other non-trivial isomorphisms (non-trivial isomorphism is what is true in addition $\mathrm{Mor~Set}$ isomorphism)
2. The subcategory $E/1$ is isomorphic to $\mathrm{Set}$ by canonical functors
3. The category is a Grothendieck topos.


Perhaps the last condition is too restrictive, then any weakening of it would be interesting. The more good categorical properties that can be preserved, the better. In fact, I don't yet see how to define such a category at all.


P.S. Of course, the condition 1 means that $f \cong f \times \mathrm{id}\_A$, if you write more carefully.



Is there a topos of quotients of sets?
I'll argue Requirement (1) and (2) together are impossible - at least not without making some highly unnatural construction. To be honest the main problem is with (1) alone.


Informally, the idea is that being of the form $E \times A \to B \times A$ is a structure, there are many way of being of this form, and each such way should corresponds to a different isomorphism with $E \to B$. And they should be different, because the corresponds to different way identifying the fibers of $E \to B$ with the fibers of $E \times A \to B \times A$. But the set of isomorphisms between $E \times A \to B \times A$ and $E \to B$ needs to be in bijection with (be a torsor under to be precise) the set of automorphisms of $ E\to B$, so there can't be that many of those. especially if we fix $E \to B$ and let $A$ vary.


Let's try to make a more formal argument - I can't completely formalize it because your requirement are a bit vague, so I'll make some assumption that we want the construction we are building to satisfies some fairly weak naturality condition.


First, I'm interpreting (1) as meaning that if you have isomorphisms $E \to E'$ and $B \to B'$ that makes the square commute then this gives an isomorphism (what you refer to as a trivial isomorphism) in your category, and that this construction should be functorial. I'm doing this because this is strongly suggested in the OP, but also because this requirement encode a minimum of requirement of naturality of the construction on sets: it just means that one does not distinguish between isomorphic sets.


So, I'm assuming you'll have a group homomorphism $TAut(E/B) \to Aut(E/B)$ where $TAut$ denotes the group of these "trivial isomorphisms" and $Aut$ is the group of automorphism in your category.


Pick $E \to B$ to be of the form $X \times B \to B$, so that it should isomorphic to $X \to 1 $ in your category. In particular $Aut(E \to B) \simeq Aut(X)$ by your requirement (2). It follows that we have a group homomorphism


$TAut(X \times B \to B) \to Aut(X)$


If I now look at the "trivial" automorphisms of $X \times B \to B$ that only acts on the $B$ component and are the identity on $X$ component, I have a subgroup of the trivial isomorphisms group isomorphic to $Aut(B)$, so such construction would produce for any sets $E$ and $B$ a choice of a group homomorphism $Aut(B) \to Aut(X)$ for any set $B$ and $X$.


The only reasonable choice here, if you want any kind of naturality for an operation doing this is that the isomorphism is trivial (And even if you want to look at non-natural things, one can talk about it, but if you take $B$ and $X$ finite with at $B$ having at least 5 elements and $X$ smaller than $B$, then the only two possibilities are trivial or factoring two the signature and an element of order $2$ of $Aut(X)$, and I'm relatively confident that one can reach a contradiction in this second case).


But then, one can simply observe that $TAut(X \times B \to B)$ identifies $Aut(X)^B \rtimes Aut(B)$, so we obtain a morphism


$Aut(X)^B \rtimes Aut(B) \to Aut(X)$


that is trivial on elements of $Aut(B)$. That is, we have a group homomorphism $Aut(X)^B \to Aut(X)$ which is invariant under permutation of the component of the products.


Here again, that operation would have to exists for any set $B$ and $X$, and it would have at the minimum to be the identity when $B = \{\*\}$. If you want any kind of naturality, this obviously cannot exists - but I'm also relatively confident that if look at it for some finite set $X$ and $B$ one can show by group-theoretic argument that no such non-trivial operation can exists.


Also, a very natural requirement in you setting would be that in the isomorphism $(E \times A \to B \times A) \simeq (E \to B)$ the trivial automorphisms of $E \to B$ corresponds to their "diagonal" action on $(E \times A \to B \times A)$ which would corresponds to the requirement that the map $Aut(X)^B \to Aut(X)$ above sends $(f,f,f,...f)$ to $f$, which is also definitely something impossible.


