Add README and results

CC-BY-4.0.txt

@@ -0,0 +1,396 @@
+Attribution 4.0 International
+
+=======================================================================
+
+Creative Commons Corporation ("Creative Commons") is not a law firm and
+does not provide legal services or legal advice. Distribution of
+Creative Commons public licenses does not create a lawyer-client or
+other relationship. Creative Commons makes its licenses and related
+information available on an "as-is" basis. Creative Commons gives no
+warranties regarding its licenses, any material licensed under their
+terms and conditions, or any related information. Creative Commons
+disclaims all liability for damages resulting from their use to the
+fullest extent possible.
+
+Using Creative Commons Public Licenses
+
+Creative Commons public licenses provide a standard set of terms and
+conditions that creators and other rights holders may use to share
+original works of authorship and other material subject to copyright
+and certain other rights specified in the public license below. The
+following considerations are for informational purposes only, are not
+exhaustive, and do not form part of our licenses.
+
+     Considerations for licensors: Our public licenses are
+     intended for use by those authorized to give the public
+     permission to use material in ways otherwise restricted by
+     copyright and certain other rights. Our licenses are
+     irrevocable. Licensors should read and understand the terms
+     and conditions of the license they choose before applying it.
+     Licensors should also secure all rights necessary before
+     applying our licenses so that the public can reuse the
+     material as expected. Licensors should clearly mark any
+     material not subject to the license. This includes other CC-
+     licensed material, or material used under an exception or
+     limitation to copyright. More considerations for licensors:
+    wiki.creativecommons.org/Considerations_for_licensors
+
+     Considerations for the public: By using one of our public
+     licenses, a licensor grants the public permission to use the
+     licensed material under specified terms and conditions. If
+     the licensor's permission is not necessary for any reason--for
+     example, because of any applicable exception or limitation to
+     copyright--then that use is not regulated by the license. Our
+     licenses grant only permissions under copyright and certain
+     other rights that a licensor has authority to grant. Use of
+     the licensed material may still be restricted for other
+     reasons, including because others have copyright or other
+     rights in the material. A licensor may make special requests,
+     such as asking that all changes be marked or described.
+     Although not required by our licenses, you are encouraged to
+     respect those requests where reasonable. More considerations
+     for the public:
+    wiki.creativecommons.org/Considerations_for_licensees
+
+=======================================================================
+
+Creative Commons Attribution 4.0 International Public License
+
+By exercising the Licensed Rights (defined below), You accept and agree
+to be bound by the terms and conditions of this Creative Commons
+Attribution 4.0 International Public License ("Public License"). To the
+extent this Public License may be interpreted as a contract, You are
+granted the Licensed Rights in consideration of Your acceptance of
+these terms and conditions, and the Licensor grants You such rights in
+consideration of benefits the Licensor receives from making the
+Licensed Material available under these terms and conditions.
+
+
+Section 1 -- Definitions.
+
+  a. Adapted Material means material subject to Copyright and Similar
+     Rights that is derived from or based upon the Licensed Material
+     and in which the Licensed Material is translated, altered,
+     arranged, transformed, or otherwise modified in a manner requiring
+     permission under the Copyright and Similar Rights held by the
+     Licensor. For purposes of this Public License, where the Licensed
+     Material is a musical work, performance, or sound recording,
+     Adapted Material is always produced where the Licensed Material is
+     synched in timed relation with a moving image.
+
+  b. Adapter's License means the license You apply to Your Copyright
+     and Similar Rights in Your contributions to Adapted Material in
+     accordance with the terms and conditions of this Public License.
+
+  c. Copyright and Similar Rights means copyright and/or similar rights
+     closely related to copyright including, without limitation,
+     performance, broadcast, sound recording, and Sui Generis Database
+     Rights, without regard to how the rights are labeled or
+     categorized. For purposes of this Public License, the rights
+     specified in Section 2(b)(1)-(2) are not Copyright and Similar
+     Rights.
+
+  d. Effective Technological Measures means those measures that, in the
+     absence of proper authority, may not be circumvented under laws
+     fulfilling obligations under Article 11 of the WIPO Copyright
+     Treaty adopted on December 20, 1996, and/or similar international
+     agreements.
+
+  e. Exceptions and Limitations means fair use, fair dealing, and/or
+     any other exception or limitation to Copyright and Similar Rights
+     that applies to Your use of the Licensed Material.
+
+  f. Licensed Material means the artistic or literary work, database,
+     or other material to which the Licensor applied this Public
+     License.
+
+  g. Licensed Rights means the rights granted to You subject to the
+     terms and conditions of this Public License, which are limited to
+     all Copyright and Similar Rights that apply to Your use of the
+     Licensed Material and that the Licensor has authority to license.
+
+  h. Licensor means the individual(s) or entity(ies) granting rights
+     under this Public License.
+
+  i. Share means to provide material to the public by any means or
+     process that requires permission under the Licensed Rights, such
+     as reproduction, public display, public performance, distribution,
+     dissemination, communication, or importation, and to make material
+     available to the public including in ways that members of the
+     public may access the material from a place and at a time
+     individually chosen by them.
+
+  j. Sui Generis Database Rights means rights other than copyright
+     resulting from Directive 96/9/EC of the European Parliament and of
+     the Council of 11 March 1996 on the legal protection of databases,
+     as amended and/or succeeded, as well as other essentially
+     equivalent rights anywhere in the world.
+
+  k. You means the individual or entity exercising the Licensed Rights
+     under this Public License. Your has a corresponding meaning.
+
+
+Section 2 -- Scope.
+
+  a. License grant.
+
+       1. Subject to the terms and conditions of this Public License,
+          the Licensor hereby grants You a worldwide, royalty-free,
+          non-sublicensable, non-exclusive, irrevocable license to
+          exercise the Licensed Rights in the Licensed Material to:
+
+            a. reproduce and Share the Licensed Material, in whole or
+               in part; and
+
+            b. produce, reproduce, and Share Adapted Material.
+
+       2. Exceptions and Limitations. For the avoidance of doubt, where
+          Exceptions and Limitations apply to Your use, this Public
+          License does not apply, and You do not need to comply with
+          its terms and conditions.
+
+       3. Term. The term of this Public License is specified in Section
+          6(a).
+
+       4. Media and formats; technical modifications allowed. The
+          Licensor authorizes You to exercise the Licensed Rights in
+          all media and formats whether now known or hereafter created,
+          and to make technical modifications necessary to do so. The
+          Licensor waives and/or agrees not to assert any right or
+          authority to forbid You from making technical modifications
+          necessary to exercise the Licensed Rights, including
+          technical modifications necessary to circumvent Effective
+          Technological Measures. For purposes of this Public License,
+          simply making modifications authorized by this Section 2(a)
+          (4) never produces Adapted Material.
+
+       5. Downstream recipients.
+
+            a. Offer from the Licensor -- Licensed Material. Every
+               recipient of the Licensed Material automatically
+               receives an offer from the Licensor to exercise the
+               Licensed Rights under the terms and conditions of this
+               Public License.
+
+            b. No downstream restrictions. You may not offer or impose
+               any additional or different terms or conditions on, or
+               apply any Effective Technological Measures to, the
+               Licensed Material if doing so restricts exercise of the
+               Licensed Rights by any recipient of the Licensed
+               Material.
+
+       6. No endorsement. Nothing in this Public License constitutes or
+          may be construed as permission to assert or imply that You
+          are, or that Your use of the Licensed Material is, connected
+          with, or sponsored, endorsed, or granted official status by,
+          the Licensor or others designated to receive attribution as
+          provided in Section 3(a)(1)(A)(i).
+
+  b. Other rights.
+
+       1. Moral rights, such as the right of integrity, are not
+          licensed under this Public License, nor are publicity,
+          privacy, and/or other similar personality rights; however, to
+          the extent possible, the Licensor waives and/or agrees not to
+          assert any such rights held by the Licensor to the limited
+          extent necessary to allow You to exercise the Licensed
+          Rights, but not otherwise.
+
+       2. Patent and trademark rights are not licensed under this
+          Public License.
+
+       3. To the extent possible, the Licensor waives any right to
+          collect royalties from You for the exercise of the Licensed
+          Rights, whether directly or through a collecting society
+          under any voluntary or waivable statutory or compulsory
+          licensing scheme. In all other cases the Licensor expressly
+          reserves any right to collect such royalties.
+
+
+Section 3 -- License Conditions.
+
+Your exercise of the Licensed Rights is expressly made subject to the
+following conditions.
+
+  a. Attribution.
+
+       1. If You Share the Licensed Material (including in modified
+          form), You must:
+
+            a. retain the following if it is supplied by the Licensor
+               with the Licensed Material:
+
+                 i. identification of the creator(s) of the Licensed
+                    Material and any others designated to receive
+                    attribution, in any reasonable manner requested by
+                    the Licensor (including by pseudonym if
+                    designated);
+
+                ii. a copyright notice;
+
+               iii. a notice that refers to this Public License;
+
+                iv. a notice that refers to the disclaimer of
+                    warranties;
+
+                 v. a URI or hyperlink to the Licensed Material to the
+                    extent reasonably practicable;
+
+            b. indicate if You modified the Licensed Material and
+               retain an indication of any previous modifications; and
+
+            c. indicate the Licensed Material is licensed under this
+               Public License, and include the text of, or the URI or
+               hyperlink to, this Public License.
+
+       2. You may satisfy the conditions in Section 3(a)(1) in any
+          reasonable manner based on the medium, means, and context in
+          which You Share the Licensed Material. For example, it may be
+          reasonable to satisfy the conditions by providing a URI or
+          hyperlink to a resource that includes the required
+          information.
+
+       3. If requested by the Licensor, You must remove any of the
+          information required by Section 3(a)(1)(A) to the extent
+          reasonably practicable.
+
+       4. If You Share Adapted Material You produce, the Adapter's
+          License You apply must not prevent recipients of the Adapted
+          Material from complying with this Public License.
+
+
+Section 4 -- Sui Generis Database Rights.
+
+Where the Licensed Rights include Sui Generis Database Rights that
+apply to Your use of the Licensed Material:
+
+  a. for the avoidance of doubt, Section 2(a)(1) grants You the right
+     to extract, reuse, reproduce, and Share all or a substantial
+     portion of the contents of the database;
+
+  b. if You include all or a substantial portion of the database
+     contents in a database in which You have Sui Generis Database
+     Rights, then the database in which You have Sui Generis Database
+     Rights (but not its individual contents) is Adapted Material; and
+
+  c. You must comply with the conditions in Section 3(a) if You Share
+     all or a substantial portion of the contents of the database.
+
+For the avoidance of doubt, this Section 4 supplements and does not
+replace Your obligations under this Public License where the Licensed
+Rights include other Copyright and Similar Rights.
+
+
+Section 5 -- Disclaimer of Warranties and Limitation of Liability.
+
+  a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
+     EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
+     AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
+     ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
+     IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
+     WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
+     PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
+     ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
+     KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
+     ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
+
+  b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
+     TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
+     NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
+     INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
+     COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
+     USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
+     ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
+     DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
+     IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
+
+  c. The disclaimer of warranties and limitation of liability provided
+     above shall be interpreted in a manner that, to the extent
+     possible, most closely approximates an absolute disclaimer and
+     waiver of all liability.
+
+
+Section 6 -- Term and Termination.
+
+  a. This Public License applies for the term of the Copyright and
+     Similar Rights licensed here. However, if You fail to comply with
+     this Public License, then Your rights under this Public License
+     terminate automatically.
+
+  b. Where Your right to use the Licensed Material has terminated under
+     Section 6(a), it reinstates:
+
+       1. automatically as of the date the violation is cured, provided
+          it is cured within 30 days of Your discovery of the
+          violation; or
+
+       2. upon express reinstatement by the Licensor.
+
+     For the avoidance of doubt, this Section 6(b) does not affect any
+     right the Licensor may have to seek remedies for Your violations
+     of this Public License.
+
+  c. For the avoidance of doubt, the Licensor may also offer the
+     Licensed Material under separate terms or conditions or stop
+     distributing the Licensed Material at any time; however, doing so
+     will not terminate this Public License.
+
+  d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
+     License.
+
+
+Section 7 -- Other Terms and Conditions.
+
+  a. The Licensor shall not be bound by any additional or different
+     terms or conditions communicated by You unless expressly agreed.
+
+  b. Any arrangements, understandings, or agreements regarding the
+     Licensed Material not stated herein are separate from and
+     independent of the terms and conditions of this Public License.
+
+
+Section 8 -- Interpretation.
+
+  a. For the avoidance of doubt, this Public License does not, and
+     shall not be interpreted to, reduce, limit, restrict, or impose
+     conditions on any use of the Licensed Material that could lawfully
+     be made without permission under this Public License.
+
+  b. To the extent possible, if any provision of this Public License is
+     deemed unenforceable, it shall be automatically reformed to the
+     minimum extent necessary to make it enforceable. If the provision
+     cannot be reformed, it shall be severed from this Public License
+     without affecting the enforceability of the remaining terms and
+     conditions.
+
+  c. No term or condition of this Public License will be waived and no
+     failure to comply consented to unless expressly agreed to by the
+     Licensor.
+
+  d. Nothing in this Public License constitutes or may be interpreted
+     as a limitation upon, or waiver of, any privileges and immunities
+     that apply to the Licensor or You, including from the legal
+     processes of any jurisdiction or authority.
+
+
+=======================================================================
+
+Creative Commons is not a party to its public
+licenses. Notwithstanding, Creative Commons may elect to apply one of
+its public licenses to material it publishes and in those instances
+will be considered the “Licensor.” The text of the Creative Commons
+public licenses is dedicated to the public domain under the CC0 Public
+Domain Dedication. Except for the limited purpose of indicating that
+material is shared under a Creative Commons public license or as
+otherwise permitted by the Creative Commons policies published at
+creativecommons.org/policies, Creative Commons does not authorize the
+use of the trademark "Creative Commons" or any other trademark or logo
+of Creative Commons without its prior written consent including,
+without limitation, in connection with any unauthorized modifications
+to any of its public licenses or any other arrangements,
+understandings, or agreements concerning use of licensed material. For
+the avoidance of doubt, this paragraph does not form part of the
+public licenses.
+
+Creative Commons may be contacted at creativecommons.org.
+

README.md

@@ -1,3 +1,67 @@
----
-license: cc-by-4.0
----
+---
+license: cc-by-4.0
+tags:
+- yahoo-open-source-software-incubator
+- action-spotting
+- soccer
+- SoccerNet
+inference: false
+---
+
+# Action Spotting in Soccer Videos Using Dense Detection Anchors
+
+This repository contains models for action spotting in soccer videos. For
+more details around the models and instructions on how to use them, please see
+https://github.com/yahoo/spivak.
+
+The technical details of our models are described in the
+following two papers, which focus on action spotting
+on the SoccerNet dataset.
+- [Temporally Precise Action Spotting in Soccer Videos Using
+Dense Detection Anchors](https://arxiv.org/abs/2205.10450).
+In ICIP, 2022.
+- [Action Spotting using Dense Detection Anchors Revisited:
+Submission to the SoccerNet Challenge 2022](https://arxiv.org/abs/2206.07846).
+arXiv preprint, 2022.
+
+Our action spotting models came in first place in the SoccerNet Challenge 2022. You can
+read more about the 2022 challenge and results in the following paper:
+- [SoccerNet 2022 Challenges Results](https://arxiv.org/abs/2210.02365).
+In MMSports, 2022.
+
+## Citations
+
+If you found our models and code useful, please consider citing our works:
+
+```
+@inproceedings{soares2022temporally,
+  author={Soares, Jo{\~a}o~V.~B. and Shah, Avijit and Biswas, Topojoy},
+  booktitle={International Conference on Image Processing (ICIP)},
+  title={Temporally Precise Action Spotting in Soccer Videos Using Dense Detection Anchors},
+  year={2022},
+  pages={2796-2800},
+  doi={10.1109/ICIP46576.2022.9897256}
+}
+
+@article{soares2022action,
+  title={Action Spotting using Dense Detection Anchors Revisited: Submission to the {SoccerNet} {Challenge} 2022},
+  author={Soares, Jo{\~a}o~V.~B. and Shah, Avijit},
+  journal={arXiv preprint arXiv:2206.07846},
+  year={2022}
+}
+```
+
+## Maintainers
+
+- Joao Soares: [email protected]
+- Avijit Shah: [email protected]
+
+## License
+
+The contents of this repository are (c) by Yahoo Inc.
+
+The contents of this repository are licensed under a
+Creative Commons Attribution 4.0 International License.
+
+You should have received a copy of the license along with this
+work. If not, see <https://creativecommons.org/licenses/by/4.0/>.

config.json

@@ -0,0 +1 @@
+"Empty config added in order to try to count the number of repo downloads."

results/spotting_challenge_validated_baidu_2.0_delta_averaged_confidence_validation_soft_nms_tuned_zoo_lr2e-3_dwd1e-3_sr0.1/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,86 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.7720031730181094, 'tight': 0.6613431726812835}
+
+loose tolerances average AP per class:
+Penalty: 0.8933245498462891
+Kick-off: 0.8361640140586659
+Goal: 0.9035236703474342
+Substitution: 0.8169946864010239
+Offside: 0.7244460264384237
+Shots on target: 0.7030747901559319
+Shots off target: 0.7083987081044311
+Clearance: 0.8223409579086662
+Ball out of play: 0.888802962559416
+Throw-in: 0.8842558039119457
+Foul: 0.8534056457396385
+Indirect free-kick: 0.692917253318859
+Direct free-kick: 0.7097838766881158
+Corner: 0.8893477461971373
+Yellow card: 0.7612362737909169
+Red card: 0.6407622763395736
+Yellow->red card: 0.3952746995013901
+
+tight tolerances average AP per class:
+Penalty: 0.8933245498462888
+Kick-off: 0.603033592209123
+Goal: 0.8626207720282714
+Substitution: 0.5892253925168659
+Offside: 0.6110643729658909
+Shots on target: 0.6503624599797839
+Shots off target: 0.6655150790454577
+Clearance: 0.6483790865169133
+Ball out of play: 0.8026450872377189
+Throw-in: 0.7758982751900265
+Foul: 0.753772167320258
+Indirect free-kick: 0.48637829800647175
+Direct free-kick: 0.6745335107255559
+Corner: 0.8403721888475367
+Yellow card: 0.6306889944031879
+Red card: 0.5968177032555326
+Yellow->red card: 0.15820240548693604
+
+Average mAP per matching tolerance:
+0.5: 0.16365755182726913
+1.0: 0.5030258788267612
+2.0: 0.6392598647802694
+3.0: 0.6847517879034775
+4.0: 0.709043465933668
+5.0: 0.721609265388677
+6.0: 0.7306311549341516
+8.0: 0.7407378127998399
+10.0: 0.7469856198826461
+15.0: 0.7672660225156853
+20.0: 0.7708444671230769
+25.0: 0.7740270271685306
+30.0: 0.7762217272183426
+35.0: 0.778106644414175
+40.0: 0.7799144536021644
+45.0: 0.7806169564449986
+50.0: 0.7822942201182824
+55.0: 0.7831801300336116
+60.0: 0.7835460039667012
+

results/spotting_challenge_validated_baidu_2.0_delta_confidence_validation_soft_nms_tuned_zoo_lr2e-3_dwd1e-3_sr0.1/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,86 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.7671254748906299, 'tight': 0.6579933646745905}
+
+loose tolerances average AP per class:
+Penalty: 0.8936369684872677
+Kick-off: 0.8338408186929629
+Goal: 0.8978844834371432
+Substitution: 0.8036849236685998
+Offside: 0.695088920234836
+Shots on target: 0.69224542713392
+Shots off target: 0.6989767046603885
+Clearance: 0.812971011030598
+Ball out of play: 0.8869790491468563
+Throw-in: 0.885126495365178
+Foul: 0.8506857249748015
+Indirect free-kick: 0.6941495477847491
+Direct free-kick: 0.6918858223653767
+Corner: 0.8871327348365177
+Yellow card: 0.7494156579469201
+Red card: 0.6506821520653727
+Yellow->red card: 0.4167466313092225
+
+tight tolerances average AP per class:
+Penalty: 0.8936369684872679
+Kick-off: 0.6139622051377991
+Goal: 0.8587725101613404
+Substitution: 0.5686737834471741
+Offside: 0.5843583546186873
+Shots on target: 0.6438704612591379
+Shots off target: 0.6543866119803063
+Clearance: 0.6420032022872344
+Ball out of play: 0.8012595564825196
+Throw-in: 0.7774267065514314
+Foul: 0.7482024576535629
+Indirect free-kick: 0.4938543676382127
+Direct free-kick: 0.653496433228923
+Corner: 0.8371987713852301
+Yellow card: 0.6157513265332809
+Red card: 0.6069038654634843
+Yellow->red card: 0.19212961715244445
+
+Average mAP per matching tolerance:
+0.5: 0.15937857627915375
+1.0: 0.5025854112197411
+2.0: 0.6368673541800091
+3.0: 0.681816778530972
+4.0: 0.7041058877579387
+5.0: 0.7157814652391425
+6.0: 0.7252736160906827
+8.0: 0.7349192353730043
+10.0: 0.7432680850156248
+15.0: 0.7612809318046297
+20.0: 0.7654585286421793
+25.0: 0.7683977594047443
+30.0: 0.7712223134872014
+35.0: 0.7733062541225844
+40.0: 0.7753989101963641
+45.0: 0.7761201157888784
+50.0: 0.7777790693431182
+55.0: 0.7786316149300495
+60.0: 0.7792518168839704
+

results/spotting_challenge_validated_resnet_normalized_delta_confidence_validation_soft_nms_tuned_zoo_lr5e-4_dwd2e-4_sr0.7/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,86 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.6710531851667352, 'tight': 0.5138127088687987}
+
+loose tolerances average AP per class:
+Penalty: 0.8634641873278239
+Kick-off: 0.7224930205040252
+Goal: 0.8952426494337323
+Substitution: 0.7994933105519891
+Offside: 0.5544295795646814
+Shots on target: 0.5886916045662123
+Shots off target: 0.6171290470542671
+Clearance: 0.7536507496187451
+Ball out of play: 0.8354714749930212
+Throw-in: 0.7962642457422582
+Foul: 0.7705223708677332
+Indirect free-kick: 0.5831696406513323
+Direct free-kick: 0.6768101837787821
+Corner: 0.8765734495911631
+Yellow card: 0.6870120225268728
+Red card: 0.30699098799197927
+Yellow->red card: 0.08049562306988109
+
+tight tolerances average AP per class:
+Penalty: 0.8229764441404981
+Kick-off: 0.3899300514304421
+Goal: 0.7911012190887683
+Substitution: 0.5004441627041327
+Offside: 0.4348476868921356
+Shots on target: 0.49466218074338975
+Shots off target: 0.5450966051627555
+Clearance: 0.5332924223991794
+Ball out of play: 0.6354967661919727
+Throw-in: 0.5152664861314659
+Foul: 0.5378079609521162
+Indirect free-kick: 0.335453381554622
+Direct free-kick: 0.6214115852150621
+Corner: 0.7725284967889736
+Yellow card: 0.49874813024382947
+Red card: 0.2791024405195717
+Yellow->red card: 0.02665003061066229
+
+Average mAP per matching tolerance:
+0.5: 0.08877974059361118
+1.0: 0.32108370750124643
+2.0: 0.4652517187878714
+3.0: 0.5421759408379384
+4.0: 0.583801422618371
+5.0: 0.6069597989607813
+6.0: 0.6205179765554759
+8.0: 0.6389499717262911
+10.0: 0.6493066637176585
+15.0: 0.6603225246783432
+20.0: 0.6666023854209809
+25.0: 0.6705269024006647
+30.0: 0.6751889030599275
+35.0: 0.6783688144997564
+40.0: 0.6801349275690669
+45.0: 0.6825193932537585
+50.0: 0.6845165412482206
+55.0: 0.6868177749159152
+60.0: 0.6876006131788097
+

results/spotting_test_baidu_2.0_delta_averaged_confidence_test_soft_nms_tuned_zoo_lr5e-4_dwd5e-4_sr0.7/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,108 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+medium: [ 1.  2.  3.  4.  5.  6.  7.  8.  9. 10.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.7901853506839145, 'medium': 0.7048231687091866, 'tight': 0.6517082287297001}
+
+loose tolerances average AP per class:
+Penalty: 0.9777175201448381
+Kick-off: 0.8236813789025365
+Goal: 0.8979735565040937
+Substitution: 0.8040031713419399
+Offside: 0.7858269225372941
+Shots on target: 0.6979313831568315
+Shots off target: 0.7123955803944721
+Clearance: 0.8450459636089352
+Ball out of play: 0.8874482348370634
+Throw-in: 0.8803127549992407
+Foul: 0.8683083263061517
+Indirect free-kick: 0.7762605030189523
+Direct free-kick: 0.7745946583206116
+Corner: 0.9000717470820573
+Yellow card: 0.8220541822761629
+Red card: 0.4666527646291228
+Yellow->red card: 0.5128723135662421
+
+medium tolerances average AP per class:
+Penalty: 0.8771026148904727
+Kick-off: 0.6477578301962404
+Goal: 0.870118821127923
+Substitution: 0.6496175654394574
+Offside: 0.7128915960551231
+Shots on target: 0.6493069198359287
+Shots off target: 0.6771916846683346
+Clearance: 0.7272252653210318
+Ball out of play: 0.8300195916516903
+Throw-in: 0.8095749779576757
+Foul: 0.802947898908119
+Indirect free-kick: 0.6376415040031789
+Direct free-kick: 0.7508939588224037
+Corner: 0.8580501809025245
+Yellow card: 0.7365973528768475
+Red card: 0.42680728754970104
+Yellow->red card: 0.3182488178495179
+
+tight tolerances average AP per class:
+Penalty: 0.8770979834308515
+Kick-off: 0.5640786739446086
+Goal: 0.8610753002668985
+Substitution: 0.5553265022958477
+Offside: 0.6436867126615218
+Shots on target: 0.6246972037981788
+Shots off target: 0.6569598540136099
+Clearance: 0.6480595854148451
+Ball out of play: 0.7808641867490481
+Throw-in: 0.7519028715526952
+Foul: 0.7515363974158611
+Indirect free-kick: 0.5390051182493727
+Direct free-kick: 0.7322030896119474
+Corner: 0.8352053507387363
+Yellow card: 0.6717969250873231
+Red card: 0.37700044120042414
+Yellow->red card: 0.20854369197313163
+
+Average mAP per matching tolerance:
+0.5: 0.15981389173834754
+1.0: 0.4871149294908528
+2.0: 0.6266745600942621
+3.0: 0.674488686601308
+4.0: 0.7020274967953382
+5.0: 0.720169413364931
+6.0: 0.7330327941159587
+7.0: 0.7415338134006204
+8.0: 0.7554614316158362
+9.0: 0.763248097839765
+10.0: 0.766429519618465
+15.0: 0.7862698358355774
+20.0: 0.7907473293900974
+25.0: 0.793910701903823
+30.0: 0.7962108703170508
+35.0: 0.7976405293110113
+40.0: 0.7986579808866683
+45.0: 0.7991671776918928
+50.0: 0.8005712828382079
+55.0: 0.8008768811842761
+60.0: 0.8029440837270495
+

results/spotting_test_baidu_2.0_delta_averaged_confidence_validation_soft_nms_tuned_zoo_lr5e-4_dwd5e-4_sr0.7/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,108 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+medium: [ 1.  2.  3.  4.  5.  6.  7.  8.  9. 10.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.7820962306021885, 'medium': 0.7114166163913449, 'tight': 0.6674191100267048}
+
+loose tolerances average AP per class:
+Penalty: 0.9288291074005357
+Kick-off: 0.7956553676771561
+Goal: 0.9188447007718313
+Substitution: 0.8281541031185021
+Offside: 0.7426178439883692
+Shots on target: 0.7005849761097851
+Shots off target: 0.700395681387768
+Clearance: 0.8241892086023699
+Ball out of play: 0.8877027976956375
+Throw-in: 0.8761964941264566
+Foul: 0.8496243692119847
+Indirect free-kick: 0.7017723372960054
+Direct free-kick: 0.7400072726021036
+Corner: 0.8874927232236671
+Yellow card: 0.7970888120133491
+Red card: 0.7891768486077261
+Yellow->red card: 0.3273032764039546
+
+medium tolerances average AP per class:
+Penalty: 0.928075038938134
+Kick-off: 0.6173334785797882
+Goal: 0.8879619021729632
+Substitution: 0.6976044258067344
+Offside: 0.6904242338451263
+Shots on target: 0.6693779807559407
+Shots off target: 0.674347152768362
+Clearance: 0.7122535560675216
+Ball out of play: 0.8389776032696287
+Throw-in: 0.8179627216936018
+Foul: 0.7977339479772989
+Indirect free-kick: 0.5659371274665296
+Direct free-kick: 0.7202067208507652
+Corner: 0.8617705226236991
+Yellow card: 0.7053049279620672
+Red card: 0.7510807424913817
+Yellow->red card: 0.15773039538332137
+
+tight tolerances average AP per class:
+Penalty: 0.9271324533601318
+Kick-off: 0.5273231405671404
+Goal: 0.8747990510394185
+Substitution: 0.6085968433347695
+Offside: 0.6348730753706364
+Shots on target: 0.6451206379556729
+Shots off target: 0.6563835020659878
+Clearance: 0.634747482098491
+Ball out of play: 0.793439376900599
+Throw-in: 0.762965711014177
+Foul: 0.7493764765700492
+Indirect free-kick: 0.4730239724203346
+Direct free-kick: 0.7081981012773698
+Corner: 0.8404242335872623
+Yellow card: 0.6482914894798508
+Red card: 0.7138079832098811
+Yellow->red card: 0.14762134020221132
+
+Average mAP per matching tolerance:
+0.5: 0.18565851507183342
+1.0: 0.5013829331198683
+2.0: 0.6449655321105975
+3.0: 0.694567563535105
+4.0: 0.7152333906149382
+5.0: 0.7284369745724891
+6.0: 0.7391045111073172
+7.0: 0.745411517934703
+8.0: 0.7500810403776189
+9.0: 0.755446471016894
+10.0: 0.7576221593850141
+15.0: 0.7761320678992906
+20.0: 0.7806705583368805
+25.0: 0.7836530811531205
+30.0: 0.7851758928789314
+35.0: 0.7891451851511624
+40.0: 0.7899686759290265
+45.0: 0.7919488199592066
+50.0: 0.7932771913993559
+55.0: 0.794008560875774
+60.0: 0.7944757127401294
+

results/spotting_test_baidu_2.0_delta_confidence_test_nms_20_zoo_lr5e-4_dwd5e-4_sr0.7/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,86 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.7769260113130184, 'tight': 0.6271998887755282}
+
+loose tolerances average AP per class:
+Penalty: 0.9550366584569878
+Kick-off: 0.8572350364226415
+Goal: 0.8982228552302304
+Substitution: 0.7891191455715266
+Offside: 0.7361963839516279
+Shots on target: 0.6642813413249665
+Shots off target: 0.696685250461912
+Clearance: 0.8345411600186557
+Ball out of play: 0.8939771171008256
+Throw-in: 0.8854899767409912
+Foul: 0.872317161014063
+Indirect free-kick: 0.7812560634277854
+Direct free-kick: 0.76136003845603
+Corner: 0.8983400482301099
+Yellow card: 0.7799941107078411
+Red card: 0.40912611532241216
+Yellow->red card: 0.49456372988270975
+
+tight tolerances average AP per class:
+Penalty: 0.8469045396195589
+Kick-off: 0.5498764962850403
+Goal: 0.8510671562210982
+Substitution: 0.5161651654694464
+Offside: 0.607035460232237
+Shots on target: 0.5734891907418288
+Shots off target: 0.6389047365263417
+Clearance: 0.6055017836175919
+Ball out of play: 0.7741193624532687
+Throw-in: 0.7281547667089718
+Foul: 0.7568971565209427
+Indirect free-kick: 0.5226693938654838
+Direct free-kick: 0.7148406575608265
+Corner: 0.8390739105934846
+Yellow card: 0.6243933000885679
+Red card: 0.29578963585528995
+Yellow->red card: 0.2175153968240013
+
+Average mAP per matching tolerance:
+0.5: 0.1412544165893352
+1.0: 0.4530648020129351
+2.0: 0.6026755818411552
+3.0: 0.6526376247162557
+4.0: 0.6753057130370195
+5.0: 0.7032964690024296
+6.0: 0.7143160196050404
+8.0: 0.7404190472406977
+10.0: 0.7513621752258413
+15.0: 0.7674850718537416
+20.0: 0.7768643186487378
+25.0: 0.7807951810891401
+30.0: 0.7834892315046657
+35.0: 0.7855187956146386
+40.0: 0.787180940995171
+45.0: 0.7880149450390048
+50.0: 0.7889339787695735
+55.0: 0.7891131876252155
+60.0: 0.7915601271525207
+

results/spotting_test_baidu_2.0_delta_confidence_test_nms_tuned_zoo_lr5e-4_dwd5e-4_sr0.7/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,108 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+medium: [ 1.  2.  3.  4.  5.  6.  7.  8.  9. 10.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.7714157823592378, 'medium': 0.6929653493880181, 'tight': 0.6426291414026194}
+
+loose tolerances average AP per class:
+Penalty: 0.9541344751582728
+Kick-off: 0.8296365146497255
+Goal: 0.8977233434622075
+Substitution: 0.7766504699385401
+Offside: 0.7288368047773323
+Shots on target: 0.6607426475351288
+Shots off target: 0.6957088500396764
+Clearance: 0.8295239628354255
+Ball out of play: 0.8908993594299848
+Throw-in: 0.8855731962517353
+Foul: 0.8689554341733067
+Indirect free-kick: 0.7680315986009753
+Direct free-kick: 0.7605163648387782
+Corner: 0.897046421612242
+Yellow card: 0.7837913805135835
+Red card: 0.39725144196101575
+Yellow->red card: 0.48904603432911387
+
+medium tolerances average AP per class:
+Penalty: 0.8552752522003082
+Kick-off: 0.6575018909753958
+Goal: 0.8646499233099091
+Substitution: 0.6232452089180528
+Offside: 0.6686785728958433
+Shots on target: 0.6062604861900417
+Shots off target: 0.6582457243283212
+Clearance: 0.7135465825706634
+Ball out of play: 0.8352274747104179
+Throw-in: 0.8219260803903096
+Foul: 0.8092376734680786
+Indirect free-kick: 0.6371941696291261
+Direct free-kick: 0.7356725977926329
+Corner: 0.8609417755438051
+Yellow card: 0.6964835393290982
+Red card: 0.3633274673964213
+Yellow->red card: 0.37299651994788185
+
+tight tolerances average AP per class:
+Penalty: 0.8469187479398719
+Kick-off: 0.5847794284551544
+Goal: 0.8509332280408388
+Substitution: 0.5366583152327312
+Offside: 0.6084317379976785
+Shots on target: 0.5830595298094111
+Shots off target: 0.6387185335112193
+Clearance: 0.6364938834737166
+Ball out of play: 0.782625623472812
+Throw-in: 0.7650808815226687
+Foul: 0.7588818613783865
+Indirect free-kick: 0.5442220802527702
+Direct free-kick: 0.7137026648443099
+Corner: 0.8392879204718685
+Yellow card: 0.6342374098013785
+Red card: 0.32092249919067806
+Yellow->red card: 0.27974105844903385
+
+Average mAP per matching tolerance:
+0.5: 0.14183609681029413
+1.0: 0.4619814177547038
+2.0: 0.6196529517653111
+3.0: 0.670902042858608
+4.0: 0.693800310480772
+5.0: 0.7103411032568689
+6.0: 0.7199973455586994
+7.0: 0.7338157425619086
+8.0: 0.7395614195151214
+9.0: 0.7442230723536972
+10.0: 0.7468068945276488
+15.0: 0.7631159731242518
+20.0: 0.7717386206139607
+25.0: 0.7747696497163885
+30.0: 0.7774259083209054
+35.0: 0.7799179624535096
+40.0: 0.7805510151041009
+45.0: 0.7807686100654511
+50.0: 0.7817636919729403
+55.0: 0.7820417422224456
+60.0: 0.7830059724031607
+

results/spotting_test_baidu_2.0_delta_confidence_test_soft_nms_tuned_zoo_lr5e-4_dwd5e-4_sr0.7/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,108 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+medium: [ 1.  2.  3.  4.  5.  6.  7.  8.  9. 10.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.7733125057378146, 'medium': 0.6947052293710451, 'tight': 0.6448624647345725}
+
+loose tolerances average AP per class:
+Penalty: 0.9475684822788822
+Kick-off: 0.8462075847366809
+Goal: 0.8982660266498871
+Substitution: 0.7851860582175508
+Offside: 0.7294598994297316
+Shots on target: 0.6600936437220334
+Shots off target: 0.6957998993736052
+Clearance: 0.8321809666742952
+Ball out of play: 0.8899925796240106
+Throw-in: 0.8864957193572609
+Foul: 0.8692191091522428
+Indirect free-kick: 0.7712606091312716
+Direct free-kick: 0.76075716915676
+Corner: 0.8972327877521046
+Yellow card: 0.786552673335563
+Red card: 0.3986286736121517
+Yellow->red card: 0.4914107153388181
+
+medium tolerances average AP per class:
+Penalty: 0.8581323768543412
+Kick-off: 0.6693975272387067
+Goal: 0.8663316546888421
+Substitution: 0.6276706469788574
+Offside: 0.6707855234215068
+Shots on target: 0.606699722577893
+Shots off target: 0.6571598644907138
+Clearance: 0.714091973314363
+Ball out of play: 0.8395593922764899
+Throw-in: 0.8250297998657812
+Foul: 0.8116689899622328
+Indirect free-kick: 0.6372958670534004
+Direct free-kick: 0.7355457448186983
+Corner: 0.8609149954310896
+Yellow card: 0.6984711320886334
+Red card: 0.3615551589244881
+Yellow->red card: 0.36967852932172857
+
+tight tolerances average AP per class:
+Penalty: 0.8530876946997727
+Kick-off: 0.5967289759028696
+Goal: 0.8548464651474659
+Substitution: 0.5430816287434017
+Offside: 0.6122424555536472
+Shots on target: 0.5855479144722032
+Shots off target: 0.6377671702106608
+Clearance: 0.6366368932500254
+Ball out of play: 0.7933395271353831
+Throw-in: 0.7702990526910746
+Foul: 0.7639325260091161
+Indirect free-kick: 0.5443199986988654
+Direct free-kick: 0.7131321554678254
+Corner: 0.8390361351890313
+Yellow card: 0.6361861238269043
+Red card: 0.31522087066549764
+Yellow->red card: 0.2672563128239891
+
+Average mAP per matching tolerance:
+0.5: 0.1759305740435704
+1.0: 0.4816021111268839
+2.0: 0.6224580178922845
+3.0: 0.6671511713838812
+4.0: 0.6930781088133451
+5.0: 0.7119230105706749
+6.0: 0.7205658006574661
+7.0: 0.7352670031414806
+8.0: 0.7411256985544997
+9.0: 0.745544071993099
+10.0: 0.7488662515384659
+15.0: 0.7661147244318718
+20.0: 0.7731920187285132
+25.0: 0.7767370727102568
+30.0: 0.7799263115613385
+35.0: 0.7814799680846098
+40.0: 0.7819293820029439
+45.0: 0.7824230635350453
+50.0: 0.7835246325474776
+55.0: 0.7837707738669264
+60.0: 0.7850237176463499
+

results/spotting_test_baidu_2.0_delta_confidence_validation_nms_20_zoo_lr5e-4_dwd5e-4_sr0.7/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,86 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.770390995770814, 'tight': 0.642914495636968}
+
+loose tolerances average AP per class:
+Penalty: 0.9066450839822355
+Kick-off: 0.8469178865110405
+Goal: 0.9038390857888022
+Substitution: 0.8000448225007153
+Offside: 0.7208766502375956
+Shots on target: 0.6612520240965962
+Shots off target: 0.6582341213813133
+Clearance: 0.8102933467667521
+Ball out of play: 0.8834902341784598
+Throw-in: 0.882516873057033
+Foul: 0.8543084810748592
+Indirect free-kick: 0.702504854445405
+Direct free-kick: 0.7061211705870223
+Corner: 0.885605990255258
+Yellow card: 0.7687829814666466
+Red card: 0.7168543355271381
+Yellow->red card: 0.3883589862469669
+
+tight tolerances average AP per class:
+Penalty: 0.9066450839822356
+Kick-off: 0.5187483463258774
+Goal: 0.8667511869438224
+Substitution: 0.5494432026707036
+Offside: 0.597874748200091
+Shots on target: 0.6000860529268568
+Shots off target: 0.6126614338399315
+Clearance: 0.5953396978494704
+Ball out of play: 0.7780359153155179
+Throw-in: 0.7348081375364344
+Foul: 0.7488452363973419
+Indirect free-kick: 0.4576028017185827
+Direct free-kick: 0.6649021911499426
+Corner: 0.8396788922228554
+Yellow card: 0.6132841194268259
+Red card: 0.6093088778868393
+Yellow->red card: 0.23553050143512824
+
+Average mAP per matching tolerance:
+0.5: 0.12200269443755349
+1.0: 0.4637662432759898
+2.0: 0.6180407271723469
+3.0: 0.6725700633657252
+4.0: 0.6922821639291424
+5.0: 0.7137638128853255
+6.0: 0.7226223322769119
+8.0: 0.7355908944052094
+10.0: 0.7473075053874547
+15.0: 0.7652876688832676
+20.0: 0.7697352894444851
+25.0: 0.7727711360332323
+30.0: 0.7746399068433374
+35.0: 0.7764688483879579
+40.0: 0.7784591889347435
+45.0: 0.7802250630849465
+50.0: 0.7803280572454051
+55.0: 0.7811238805513965
+60.0: 0.7821450044801306
+

results/spotting_test_baidu_2.0_delta_confidence_validation_nms_tuned_zoo_lr5e-4_dwd5e-4_sr0.7/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,108 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+medium: [ 1.  2.  3.  4.  5.  6.  7.  8.  9. 10.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.7640375170036238, 'medium': 0.6996453876391838, 'tight': 0.6554480665245292}
+
+loose tolerances average AP per class:
+Penalty: 0.9065799550691592
+Kick-off: 0.8011743667823625
+Goal: 0.903586224985416
+Substitution: 0.7931629698435598
+Offside: 0.7115807878441616
+Shots on target: 0.6589001434359805
+Shots off target: 0.6557453269926699
+Clearance: 0.8022458092997428
+Ball out of play: 0.8847730873373376
+Throw-in: 0.880504948523891
+Foul: 0.8521327048657309
+Indirect free-kick: 0.690544618225469
+Direct free-kick: 0.7007580550567019
+Corner: 0.8843731145790283
+Yellow card: 0.7669152610146522
+Red card: 0.706846854078587
+Yellow->red card: 0.3888135611271544
+
+medium tolerances average AP per class:
+Penalty: 0.9065799550691589
+Kick-off: 0.6337769437016447
+Goal: 0.8828584709179004
+Substitution: 0.6527245662085706
+Offside: 0.6567983642169009
+Shots on target: 0.6290778865770644
+Shots off target: 0.631164902315359
+Clearance: 0.6986052093079859
+Ball out of play: 0.8373141541849317
+Throw-in: 0.8263900526382705
+Foul: 0.8012854468053682
+Indirect free-kick: 0.5662690634766823
+Direct free-kick: 0.6759094385474593
+Corner: 0.8601393245172576
+Yellow card: 0.6876351823687716
+Red card: 0.6775338662291612
+Yellow->red card: 0.2699087627836383
+
+tight tolerances average AP per class:
+Penalty: 0.9065799550691591
+Kick-off: 0.5465793220597293
+Goal: 0.8667854002561278
+Substitution: 0.5664697509347744
+Offside: 0.6000051641577088
+Shots on target: 0.6057113400997751
+Shots off target: 0.6139938357214235
+Clearance: 0.6232045548951356
+Ball out of play: 0.7869996430575694
+Throw-in: 0.771317045775858
+Foul: 0.7519312298872178
+Indirect free-kick: 0.47818838817726406
+Direct free-kick: 0.6600378393846589
+Corner: 0.839896114128786
+Yellow card: 0.6264190256409359
+Red card: 0.6408926314173791
+Yellow->red card: 0.257605890253493
+
+Average mAP per matching tolerance:
+0.5: 0.12197970096601987
+1.0: 0.4745429721261379
+2.0: 0.6337733567895615
+3.0: 0.6858684645772504
+4.0: 0.7051502930732096
+5.0: 0.7194573311900525
+6.0: 0.7278921246564946
+7.0: 0.7333691979296498
+8.0: 0.7392865241955524
+9.0: 0.7423616853105406
+10.0: 0.7447560499345491
+15.0: 0.7593549216974149
+20.0: 0.7632141239010325
+25.0: 0.7657749103994403
+30.0: 0.7673726794495715
+35.0: 0.7693732837193636
+40.0: 0.7704546418977203
+45.0: 0.7713058231125371
+50.0: 0.77239994301977
+55.0: 0.7735019137465673
+60.0: 0.7743514611337374
+

results/spotting_test_baidu_2.0_delta_confidence_validation_soft_nms_tuned_zoo_lr5e-4_dwd5e-4_sr0.7/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,108 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+medium: [ 1.  2.  3.  4.  5.  6.  7.  8.  9. 10.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.7666546711359695, 'medium': 0.7008803934439402, 'tight': 0.6574110111503904}
+
+loose tolerances average AP per class:
+Penalty: 0.9067847439450063
+Kick-off: 0.8253936460499266
+Goal: 0.9042830163085928
+Substitution: 0.8024082807553004
+Offside: 0.7133479733237874
+Shots on target: 0.6574991548869498
+Shots off target: 0.6560153379529499
+Clearance: 0.8066662866730162
+Ball out of play: 0.8862699707961642
+Throw-in: 0.8810632484453382
+Foul: 0.8526824482602198
+Indirect free-kick: 0.6939249471981042
+Direct free-kick: 0.696524868992775
+Corner: 0.8845122853248307
+Yellow card: 0.7723299331210148
+Red card: 0.7074737026919778
+Yellow->red card: 0.38594956458552976
+
+medium tolerances average AP per class:
+Penalty: 0.9067847439450062
+Kick-off: 0.6475158356219493
+Goal: 0.885842621040736
+Substitution: 0.6558868864134699
+Offside: 0.660024013901094
+Shots on target: 0.6289225130895878
+Shots off target: 0.629837280725976
+Clearance: 0.7002349384853828
+Ball out of play: 0.8422490693735313
+Throw-in: 0.8278494977116577
+Foul: 0.8041911236430039
+Indirect free-kick: 0.566070087612474
+Direct free-kick: 0.6740032171108139
+Corner: 0.8598906588222671
+Yellow card: 0.6897106301526662
+Red card: 0.6726502221628576
+Yellow->red card: 0.2633033487345103
+
+tight tolerances average AP per class:
+Penalty: 0.9067847439450063
+Kick-off: 0.5554885655674314
+Goal: 0.8726287488785371
+Substitution: 0.5682501670257499
+Offside: 0.6048617651061438
+Shots on target: 0.6079284816326378
+Shots off target: 0.6122937686799483
+Clearance: 0.6232704535007372
+Ball out of play: 0.7990485645882682
+Throw-in: 0.7734663047431465
+Foul: 0.7577964289771999
+Indirect free-kick: 0.4767528903773843
+Direct free-kick: 0.660428405637004
+Corner: 0.8398081447082903
+Yellow card: 0.6321654052089108
+Red card: 0.6291208715014576
+Yellow->red card: 0.255893479478782
+
+Average mAP per matching tolerance:
+0.5: 0.1563080993304258
+1.0: 0.49085027311497725
+2.0: 0.6357103820810186
+3.0: 0.6839774250065324
+4.0: 0.7048645148073858
+5.0: 0.7193331722982718
+6.0: 0.7289971595913448
+7.0: 0.7340860393670355
+8.0: 0.7386448565538268
+9.0: 0.7439313899353948
+10.0: 0.7459069295943266
+15.0: 0.761347238921781
+20.0: 0.7652571673851115
+25.0: 0.7680801785167116
+30.0: 0.7698210214782697
+35.0: 0.7720551993546106
+40.0: 0.7729473665917244
+45.0: 0.7756226483083732
+50.0: 0.7766744898993755
+55.0: 0.7769245937745075
+60.0: 0.7777959250434758
+

results/spotting_test_resnet_normalized_delta_confidence_test_soft_nms_tuned_zoo_lr1e-3_dwd5e-4_sr0.5/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,108 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+medium: [ 1.  2.  3.  4.  5.  6.  7.  8.  9. 10.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.713592281581524, 'medium': 0.6034959413608697, 'tight': 0.5315459255066107}
+
+loose tolerances average AP per class:
+Penalty: 0.9542739192019869
+Kick-off: 0.7464836368577535
+Goal: 0.899442061453813
+Substitution: 0.7606884301574484
+Offside: 0.6892383654674122
+Shots on target: 0.5915579178738924
+Shots off target: 0.6234708614771872
+Clearance: 0.7991533740936424
+Ball out of play: 0.8607637641597311
+Throw-in: 0.8107463337784017
+Foul: 0.8125639748136317
+Indirect free-kick: 0.6930779597895534
+Direct free-kick: 0.7322949211950888
+Corner: 0.8961033620198063
+Yellow card: 0.7708468167121472
+Red card: 0.18190797777405773
+Yellow->red card: 0.30845511006035237
+
+medium tolerances average AP per class:
+Penalty: 0.8279492034771396
+Kick-off: 0.526198334485767
+Goal: 0.8461083479667999
+Substitution: 0.5794428615984838
+Offside: 0.604389492011896
+Shots on target: 0.5245169716904665
+Shots off target: 0.5813239847860032
+Clearance: 0.6618745929142972
+Ball out of play: 0.7364201149326012
+Throw-in: 0.6382241597562968
+Foul: 0.6666375405686371
+Indirect free-kick: 0.49978464482611007
+Direct free-kick: 0.6917992822451839
+Corner: 0.823257284723533
+Yellow card: 0.6473915840662481
+Red card: 0.17468323870400243
+Yellow->red card: 0.22942936438131944
+
+tight tolerances average AP per class:
+Penalty: 0.795588501021627
+Kick-off: 0.42039631936109617
+Goal: 0.7971925502933921
+Substitution: 0.4578624667087565
+Offside: 0.5271980646150041
+Shots on target: 0.47995093736159
+Shots off target: 0.5521967611756651
+Clearance: 0.5679998776343989
+Ball out of play: 0.634226802804583
+Throw-in: 0.5092852075634913
+Foul: 0.5465529136268927
+Indirect free-kick: 0.3916086361586705
+Direct free-kick: 0.6561794068958066
+Corner: 0.7626622078845513
+Yellow card: 0.572083503548805
+Red card: 0.16874558795025807
+Yellow->red card: 0.19655098900779444
+
+Average mAP per matching tolerance:
+0.5: 0.08757177035063746
+1.0: 0.3301789375761382
+2.0: 0.4835272754533661
+3.0: 0.5611419547155474
+4.0: 0.6029325020329863
+5.0: 0.6269850020729479
+6.0: 0.6459848837154986
+7.0: 0.6577344901597738
+8.0: 0.6679281168363027
+9.0: 0.6782839993670017
+10.0: 0.683711558212668
+15.0: 0.7061205095834024
+20.0: 0.7128835946645163
+25.0: 0.7170246197736503
+30.0: 0.7202737872783705
+35.0: 0.7223756389497731
+40.0: 0.724124513081797
+45.0: 0.7256697156311506
+50.0: 0.7274332633967501
+55.0: 0.7301457489711495
+60.0: 0.7325192936341238
+

results/spotting_test_resnet_normalized_delta_confidence_validation_soft_nms_tuned_zoo_lr1e-3_dwd5e-4_sr0.5/Evaluation/evaluation_aggregate.txt

@@ -0,0 +1,108 @@
+Spotting evaluation:
+
+Classes used when computing mAP:
+Penalty  
+Kick-off  
+Goal  
+Substitution  
+Offside  
+Shots on target  
+Shots off target  
+Clearance  
+Ball out of play  
+Throw-in  
+Foul  
+Indirect free-kick  
+Direct free-kick  
+Corner  
+Yellow card  
+Red card  
+Yellow->red card  
+
+Tolerances used when computing averages:
+loose: [ 5. 10. 15. 20. 25. 30. 35. 40. 45. 50. 55. 60.]
+medium: [ 1.  2.  3.  4.  5.  6.  7.  8.  9. 10.]
+tight: [1. 2. 3. 4. 5.]
+
+Average-mAP: {'loose': 0.703927703670766, 'medium': 0.6041538907909546, 'tight': 0.5330582045113054}
+
+loose tolerances average AP per class:
+Penalty: 0.9509748237020966
+Kick-off: 0.7259552181565746
+Goal: 0.9095116729284654
+Substitution: 0.7936337124389334
+Offside: 0.6480643122134687
+Shots on target: 0.6068083843579278
+Shots off target: 0.6230070474808825
+Clearance: 0.7790300020238015
+Ball out of play: 0.8596121691681573
+Throw-in: 0.8179215946787723
+Foul: 0.7889245173777065
+Indirect free-kick: 0.6178282714361609
+Direct free-kick: 0.7002606353358407
+Corner: 0.8915350238365994
+Yellow card: 0.7352854522237819
+Red card: 0.39576191562147484
+Yellow->red card: 0.12265620942237676
+
+medium tolerances average AP per class:
+Penalty: 0.9408168239326865
+Kick-off: 0.4956149071479594
+Goal: 0.8585387347865504
+Substitution: 0.6186881927290516
+Offside: 0.575947605237714
+Shots on target: 0.55303175475626
+Shots off target: 0.5855608663268299
+Clearance: 0.6380264082155939
+Ball out of play: 0.7479698042007985
+Throw-in: 0.6558316848949423
+Foul: 0.654243157100982
+Indirect free-kick: 0.4465742623371977
+Direct free-kick: 0.6587929490448693
+Corner: 0.8328336252869152
+Yellow card: 0.6163677995046489
+Red card: 0.3496521700115829
+Yellow->red card: 0.04212539793164527
+
+tight tolerances average AP per class:
+Penalty: 0.9281193242209242
+Kick-off: 0.38001034204852746
+Goal: 0.8139131378932276
+Substitution: 0.5040213924520766
+Offside: 0.5007822167085725
+Shots on target: 0.5082035319954923
+Shots off target: 0.551905061941975
+Clearance: 0.5511757617794517
+Ball out of play: 0.6428607471658025
+Throw-in: 0.5250318842076994
+Foul: 0.5363858681150431
+Indirect free-kick: 0.3446238125370218
+Direct free-kick: 0.6270378189541572
+Corner: 0.7795332541905443
+Yellow card: 0.5428300593961923
+Red card: 0.3190540549262046
+Yellow->red card: 0.006501208159277306
+
+Average mAP per matching tolerance:
+0.5: 0.10342681098596915
+1.0: 0.3386610331826849
+2.0: 0.48568618894689913
+3.0: 0.5616148316649907
+4.0: 0.6009099170558765
+5.0: 0.6293827275722248
+6.0: 0.6456241096754219
+7.0: 0.6567451181647525
+8.0: 0.6708868272935067
+9.0: 0.6765533007070913
+10.0: 0.681302958892971
+15.0: 0.6934582774695656
+20.0: 0.7002662170619169
+25.0: 0.7034815833462462
+30.0: 0.7074725030818622
+35.0: 0.7120320157733794
+40.0: 0.7133607949047807
+45.0: 0.7159838004530386
+50.0: 0.7188563704677908
+55.0: 0.7213244182527297
+60.0: 0.7219488737760661
+