 If you haven't already, you'll need to set up a Python environment with the latest versions of PyTorch and TensorBoard. The commands on screen show how to do that for Conda and Pip. We'll also be using matplotlib to manipulate images. Once you have the dependencies installed, you can run the companion notebook for this video in the environment you set up. For this model, we're going to be training a simple neural network to recognize different articles of clothing. We'll visualize data elements directly, track the success of the training process, we'll use TensorBoard to look under the hood at the model itself, and we'll do a more advanced visualization of the dataset as a whole and its internal relationships. For a dataset, we'll use Fashion MNIST. This is a set of small image tiles that depict various garments, classified by the type of garment depicted. For a model, we'll use a version of Linnet5 tweaked to accommodate the FashionMNIST dataset. We'll start by importing the libraries we need and the SummaryWriter class from torch.utils.tensorboard. This is the class wrapping the TensorBoard support in PyTorch and will be your primary interface for interacting with TensorBoard. It's good practice to visualize your training data prior to feeding it to your model, especially with computer vision tasks. Let's set up our data set. We'll use TorchVision to download training and validation splits of the data set. We'll talk more about validation shortly. And we'll also set up data loaders for each of the data set splits and define the classes against which we're classifying. Let's visualize a few instances of the data set. We'll use an iterator to pull out a few instances of the data and create a matplotlib helper function to batch them together in a grid. Let's show them in the notebook. So how do we add this to TensorBoard? It's a one-liner to write the data to the log directory. Note that we also call flush on our summary writer object. This makes sure that everything we've logged through the writer has been written to disk. Now let's switch to a terminal and start TensorBoard. We'll copy the URL that the TensorBoard command line gave us and look at the Images tab. Note here that the image we've added has a header containing the label we applied when we saved the image to the TensorBoard log directory. Next, we'll use TensorBoard to help assess our training process. We'll graph the accumulated training loss for regular time steps and compare it to the loss measured against a validation data set. For background, here's a brief aside on what we're doing and why. If you took a math class, it's likely that you'd be given homework problem sets to solve. After a number of homework sets, you'd be given an exam. The exam problems would be similar in nature but different in their specifics to the homework problems you've seen already. This is intended to make sure you learn the content of the class and not just memorize the homework problems. Similarly, we can use a validation dataset, that is, a portion of the total dataset not used for training, to see whether our model is learning generally or whether it's overfitted to the training data, akin to memorizing the training instances instead of modeling the general function we're trying to optimize the model for. Let's set up a training loop with validation checks and graph the results. Here we have a training loop. You can see that at the top of the code, we've declared a variable to accumulate the measured loss of the model's predictions, which will report every thousand training steps. We'll also be doing a separate loss check against the validation dataset. For tracking and comparing two different quantities, we use the add scalars call on Summary Writer, which allows us to add a dictionary containing multiple scalar values, each with distinct tags that get their own line on the graph. Let's run the cell and see what that looks like. Switching over to TensorBoard and looking at the Scalars tab, we can see that our loss is decreasing monotonically over the training run. This is a nice reassurance that the training is working. But are we overfitting? Looking at the graph, we can see that the validation and training curves are converging nicely. Next, let's use TensorBoard to better understand our model and how data flows through it. To do this, we'll use the add graph method on our summary writer. This method takes as arguments the model and a sample input that will be used to trace data flow through the model. We'll run the cell and switch over to TensorBoard. And going to the graphs tab, we can see a very simple graph showing the model with inputs going in one side and output submitted from the other. Of course, we'll want more detail in this, and we can get it by double-clicking on the model node in the graph. And here we can see a graph containing all of our layers and arrows indicating how data flows through them. Note that because the model uses the same maxpool object twice, the second convolutional layer appears to be embedded in a loop. But as you can see from the code, the flow is more linear than that. We've already used TensorBoard to display visualizations of instances of our data, but what about the whole dataset? An embedding is a mapping of instances from a higher dimensional space to a lower dimensional one. This is a common technique in NLP. If you have a 10,000 word vocabulary, represented by one-hot vectors, your words are unit vectors in a 10,000-dimensional space. If you train an embedding layer that maps these vectors to a lower-dimensional space, relationships can emerge. For example, the new vectors for words like good, excellent, and fabulous will tend to be clustered in that lower-dimensional space. In our case, our 28x28 image tiles can be thought of as 784 dimensional vectors. We can use the Summary Writer's Add Embedding method to project this down to an interactive 3D visualization. Here's a bit of code to select a random sample of our data, label it, and project it. Note that as always, we use the flush method to ensure that all our data is written to disk. Switching over to TensorBoard, we can see on the Projector tab a 3D visualization of our new embedding. Zoomed out, we can see some large structures, some arcs within the 3D space. Zooming in on some of these structures, we can see that some of these arcs have clustered similar garment types. Zoom in on your own sample of the data and see if you can identify patterns in how different types of garments are clustered in this 3D projection. For more information on PyTorch's TensorBoard support, you can visit the PyTorch documentation at pytorch.org for a full documentation of torch.utils.tensorboard.summarywriter. The PyTorch tutorials section at pytorch.org has tutorials on using TensorBoard. And the TensorBoard documentation, of course, has more detail about TensorBoard itself if you want a deeper view of what the summary writer is doing under the hood.
PyTorch TensorBoard Support
TensorBoard is a well-known tool for visualizing the activity of machine learning models and training progress. PyTorch provides support for TensorBoard use, with no dependency on other ML frameworks. This video will show you how to visualize graphs of training progress, input images, model structure, and dataset embeddings with TensorBoard and PyTorch.

Download the TensorBoard Support notebook here: https://pytorch-tutorial-assets.s3.amazonaws.com/youtube-series/Video+5+-+Tensorboard+Support+in+PyTorch.ipynb