Internet Technical Bookshop

Books and Reources for the Sciences and the Arts

The NVidia Jetson TX2 - Truly a Big Beast in the Embedded Systems World

In late 2016 I was invited by the Vlatacom research group in Belgrade to run a very intensive 5 day real time Linux programming workshop. As well as the usual Posix API programming and Kernel module and device driver programming we also explored Xenomai real time programming and how it was used in the Bela cape developed at Queen Mary University of London for developing new interactive electronic musical instruments. It was my first visit to Belgrade a beautiful city that is recovering economically after the troubles resulting from the break up of the former Yugoslavia, and I was introduced to the delights of Serbian cuisine. This year I have been invited, again, to deliver another advanced workshop. The theme of this year's workshop will be the NVidia Jetson TX2 and running advanced Linux applications on it.

Sometimes you can even purchase an NVidia Jetson TX2 development kit on Amazon. However, if you are working in an academic environment it is worth noting that NVidia has an academic program which can save you money.

The NVidia Jetson TX2 is truly a powerful beast and one of a number of such beasts that can be used for applications such as autonomous car driving systems and UAVs as well as next generation surveillance, security and industrial automation applications. Compared to the Jetson TX2 the RaspberryPi 3 and the BeagleBone Black are but minnows. The GPU processing engine on the Jetson TX2 has 256 GPU cores, and the Jetson TX2 has four 64bi ARM cores in addition. No surprise, then, to learn that Serious machine learning applications can be run on the Jetson TX2 as well as fast OpenCV based feature recognition and image processing applications. A follow on development from preparing this workshop has been the development of a set of courses exploring various aspects of application development using the Jetson TX2 and related processors. These will shortly be appearing on the First Technology Transfer web site. Preparing for the workshop has required sifting through a ton of resources, including technical books dealing with CUDA, Machine Learning using Neural Networks, Tensor flow, OpenCV, OpenCL and OpenMP. In this scribend I'll take a look at various books covering CUDA, and Machine Learning using Neural Networks. In a follow on Scribend I hope to cover Open CV , Open MP and Real Time Linux.

Although CUDA is a relatively open standard, in practice it is associated with NVidia. The development of affordable high performance GPUs from NVidia has truly revolutionised machine learning using Neural Networks. In one of his lectures towards the end of the Coursera Neural Network course on Coursera Geoffrey Hinton, who has been involved with Neural Network research for several decades argues that the recent progress in deep machine learning using neural networks has been because of the availability of powerful parallel computing engines such as those found in NVidia GPU cards, and now, in the Jetson TX2. Connecting together 10 Jetson TX2s over PCIe by means of a high performance PCIe switch results in a computer having over 2000 GPU processor cores, as well as 40 64bit ARM cores. With clever engineering and judicial selection of processing problems this is, essentially, a supercomputer coming in at well under $25,000. The challenge is how to make the most of all this potential power.

Parallel computation in CUDA and neural network computing is neither trivial or easy. Deep machine learning relatively rarely "turns up gold". However some of the outstanding successes serve to "spur on continued research". To get an idea of the kind of programming skills need to "become a CUDA virtuoso" take a look at the lectures in this year's Caltech GPU programming course http://courses.cms.caltech.edu/cs179/ In particular, before starting take a look at the lab 1 assignment which sets the tone as follows: This class will make heavy use of low-level C constructs and concepts, especially pointers and memory management. As a "warm-up", here are a few quick samples of code and their intended specifications. Each such piece of code is incorrect. Identify what is wrong with the code, and how it should be fixed. (Many of these problems allude to common errors encountered while writing both GPU and CPU code.) Some of the later labs in this course are quite challenging, as are most of the lectures in the second half of the course. The set book for the course, by is CUDA Handbook: A Comprehensive Guide to GPU Programming. This is a tough book, and not for the faint hearted. As one of the reviewers of this book on Amazon put it, regarding memory "There are however some major problems with this book. I am not sure author gave someone this book to read it. You learn about memory in chapter 2. Then you learn about memory in chapter 3. If you haven't had enough, the whole chapter 5 is about memory." ... You have been warned.

CUDA for Engineers: An Introduction to High-Performance Parallel Computing is a bit gentler, and more practical in nature. The emphasis is more on how to use CUDA to tackle engineering problems such as e.g.

  • Visualizing functions in 2D and 3D
  • Solving differential equations while changing initial or boundary conditions
  • Viewing/processing images or image stacks
  • Computing inner products and centroids
  • Solving systems of linear algebraic equations
  • Monte-Carlo computations
Some of these applications, such as viewing and processing images, are more relevant to Jetson TX2 embedded systems applications than some of the others, such as e.g. Monte-Carlo computations.

Geoffrey Hinton's Neural Networks Coursera course was Matlab oriented. It was a tough course (I should know, I took it), but most enjoyable, including fixing a tricky bug in one of the assignments, which caused a bit of consternation. Some of the assignments could have been attempted in Python. There are some useful Neural Network programming in Python books, and, starting off on Neural Networks using Python Neural Network computing frameworks makes good sense. Tariq Rashid's Make Your Own Neural Network is a relatively gentle introduction to the subject that lays good practical foundations. For a relatively complete overview of AI, Deep Learning and Neural Networks John Slavio's book /Deep Learning and Artificial Intelligence: A Beginners’ Guide to Neural Networks and Deep Learning has much to recommend it. It makes good use of analogies to real world examples to help explain some of the more technical aspects of working with neural networks. Although not a programming book it does introduce a library/framework approach that can be used to implement a custom neural network. It would also be a good book to have in the A Level Computer Science section of a school or college library. A gentle programming oriented introduction to Neural Networks using Python is Michael Taylor's Make Your Own Neural Network: An In-depth Visual Introduction For Beginners. This book could form the basis of some interesting A Level computer science projects for able and motivated 6th formers. Certainly, if I was interviewing a University applicant and they could talk knowledgeably about Neural Networks at the level of this book I would be favourably impressed. Because Python is one of the core languages in GCSE and A Level computer science this book, though moderately challenging, can be suggested as extra-curricular reading for the more enthusiastic members of an A Level computer science class. If this Scribend is being read by a parent or A level computer science student interested in some coaching/tutoring/mentoring in this area, and living in South London then a visit to A href="carshaltontutorialcollege.london">Carshalton Tutorial College might be in order.

For more cutting edge Neural Network programming C and C++ are the languages of choice. If looking for a thorough coverage with examples we are in luck. Timothy Masters has published a three volume compendium covering the major topics associated with neural network C++ programming, using, conveniently, C++ and CUDA.

For a good reference book on Neural Network design that is Matlab oriented I can heartily recommend Neural Network Design (2nd Edition) by Martin Hagan and colleagues. I found it most helpful when studying Geoffrey Hinton's Coursera Course. Also, important when looking into cutting edge techniques in deep machine learning is the whole area of Tensor Flow computing. Wikipedia contextualises TensorFlow as "Google Brain's second generation system. Version 1.0.0 of which was released on 11 February 2017 ... TensorFlow can run on multiple CPUs and GPUs, and there are CUDA extensions for general-purpose computing on graphics processing units. TensorFlow computations are expressed as stateful dataflow graphs, and the name TensorFlow itself derives from the operations that such neural networks perform on multidimensional data arrays which are referred to as "tensors". A "dip your toe in the water" book on Tensor flow is First Contact With Tensorflow by Jordi Torres. Sooner or later most scientists and engineers will have to grapple with Machine Learning, and this will, almost certainly, include having to "wrestle with" Tensor Flow. This book is, currently, a very useful introduction to the subject for practically minded types. However, Tensor flow needs to be seen in context. In my opinion, a classic, reference to advanced neural network machine learning is Deep Learning (Adaptive Computation and Machine Learning Series) by Ian Goodfellow, Yoshua Bengio and Alain Courville all of whom make an appearance in Professor Hinton's course, and, I think, have been much influenced by him. As Elon Musk, cochiar of OpenAI, put it "Written by three experts in the field, Deep Learning is the only comprehensive book on the subject.". If you do not have a much thumbed copy then order it and "get studying". A more advanced book, this one is Python oriented, is Hands on Machine Learning with Scikit-Learn and Tensorflow: Concepts, Tools, and Techniques for Building Intelligent Systems by Aurelien Geron. It is currently, probably one of the best introductions to serious Machine Learning in the Pythonic Machine Learning ecosystem, and this is not just my opinion. NVidia does provide libraries for Tensor Flow based applications on its GPUs, and the code can run on the Jetson TX2.

There are no books out yet on Jetson TX2 programming. However, there are some really passionate and gifted enthusiasts who hang out on the JetsonHacks website. http://www.jetsonhacks.com/ This is the place to go to find out what exciting things devices such as the Jetson TX2 are being used for. such as

If you visit the JetsonHacks site and are not "inspired" then, sadly, maybe this whole field of technology is "not for you". Underlying many advanced robotics projects is ROS (Robot Operating System) ... of which more in another Scribend. If you are interested in technical training on various aspects of the NVidia Jetson family then take a look at the NVidia courses offered by First Technology Transfer.