code: private static int calculateMaskPenalty(ByteMatrix matrix) {
      return MaskUtil.applyMaskPenaltyRule1(matrix)
          + MaskUtil.applyMaskPenaltyRule2(matrix)
          + MaskUtil.applyMaskPenaltyRule3(matrix)
          + MaskUtil.applyMaskPenaltyRule4(matrix);
    }
docs: Basically it applies four rules and summate all penalties.