I have 3 neural networks processing 3 different vectors of values. Each NN processes a sample of it's vector and gives binary result (y/n) that is correct with given probability. All 3 NNs give answer to the same question. The task is to combine this results into single one (y/n) and figure out probability that it is correct.


What methods can you suggest for this task?


I found random effects model, but i can't apply it to my task because it operates with means and variances while my data is different: it's a binary answer with probability of correctness.


Thanks in advance.



Combine results with different veracity
Assume that the networks err independently and the corresponding probabilities are $(TP)\_j, (TN)\_j, (FP)\_j, (FN)\_j$ (false negative = the answer is yes and the network outputs no, etc.). Also assume that the price you pay for an error of each type is the same for both answers.


Suppose that you get YNY, say. This means either Yes and $(TP)\_1(FN)\_2(TP)\_3$, or No and $(FP)\_1(TN)\_2(FP)\_3$. The combined output should correspond to the higher product, taking into account the prevalence $P$ of the Yes answer, so if you have $P(TP)\_1(FN)\_2(TP)\_3>(1-P)(TP)\_1(FN)\_2(TP)\_3$, you settle on Yes, otherwise on No. 


You cannot beat the (properly understood) maximal likelihood under any circumstances and this case is simple enough to match it if you know the prevalence and all 3 two by two detection tables. After that it is easy to figure out the detection table for the combined method by a straightforward casework. 


