I have seen that some papers talk of computational cost of the network and they measure it in MACs. I didn't find any clear explanation of what it is.


Could anyone explain in clear words the meaning of computational cost and why it should be taken into consideration in a network?



What is the computational cost in a neural network?
Computational cost is simply a measure of the amount of resources the neural network uses in training or inference, which is important so you can know how much time or computing power you'll need to train or use an NN. It can measured in a variety of ways, but common ones are time and number of computations, expressed either as number of floating point operations (FLOPs) or as number of multiply-and-accumulate operations (MACs or MACCs). Since a lot of what a neural network is doing will be multiplying inputs by weights and adding them together, this is obviously a useful measure.


For a survey of the topic I'd recommend [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://arxiv.org/abs/1703.09039).


