9 HN users recommended this talk7 months2011–2013score 9.5
Years with an HN recommendation, 2007–2026
HN commenters recommended it for years as a simple, exciting overview of the deep learning wave.
Andrew Ng presents unsupervised feature learning and deep learning, showing how algorithms can learn representations from unlabeled data instead of hand-designed features. He gives examples in vision and audio, including visualizations of features learned by successive layers.
“he explains very simply how all this works for images and some examples about the audio case.” (HN, 2012)
A good intro page with lots of links: [link] A lot of the newest work is under a few different names: "deep learning", "convolutional deep networks", "unsupervised feature learning", etc. Here's a great talk by Andrew Ng of Stanford, who's a recent convert to this area: [this video] (Note that this is quite a one-sided view of things, but it does convey the excitement of deep-learning researchers and the potential of what might be possible.) Although I'm not in deep learning myself (I'm a computer vision researcher), here's a TL;DR as I understand it: rather than having people in specific domains such as computer vision or speech processing create their own features, the idea is to take raw inputs (pixels in the case of images) and train multi-layer neural net architectures that "learn" the relevant higher-level features in an unsupervised way (i.e., without labeled training data). Some of these seem to be pulling out interesting features and perform competitively on a few benchmarks in vision and other fields. I'm not sold on this yet, because it seems like the complexity of designing features has merely been traded in for the complexity of designing different learning architectures, but it's certainly becoming quite popular these days (mostly led by Geoff Hinton of Toronto, Yoshua Bengio of Montreal, and Yann LeCunn of NYU).
Conveys the excitement and potential of unsupervised feature learning.
This seems very related to this [this video] speak by Andrew Ng. It is a 40min speak, but he explains very simply how all this works for images and some examples about the audio case. It is incredible how using this deep learning techniques we can teach this "neural networks" to recognize such complicated patterns. It is like reverse engineering the brain's algorithms. BTW I took his Coursera's course about Machine Learning and it was great! I also recommend it A LOT to gather basic ML knowledge.
Simple 40 minute explanation of deep learning for images and audio.
Check out this (awesome) talk: [this video] There are a number of areas in which unsupervised feature learning outperforms the hand-programmed models we have been fine-tuning for decades.
the one overview I've found the most useful is [this video] (Bay Area Vision Meeting: Unsupervised Feature Learning and Deep Learning by Andrew Ng in April 2011).
Most useful overview of unsupervised feature learning.
Andrew Ng has a fantastic example of visualizing deep belief nets: [this video] The example starts at about 18:20. You really need to have watched the previous material to understand it, but the basic idea is that he's plotting the hierarchy of features learned by successive layers of a deep neural net (a sparse autoencoder, IIRC, it's been a while since I watched it).
Visualizes hierarchy of features learned by successive deep net layers.
They kind of hint at it in the article, but sounds like automatic feature detection really helped here. That's really the future (or present): learning algos discovering better features automatically, significantly cutting down the labor involved in feature engineering. Pretty neat. Here's a video talking about current research into this at Stanford: [this video]
Current Stanford research on automatic feature learning.
Andrew Ng has set many state of the art results on various data sets using similar approaches as the one described in the paper. Here is a reasonably approachable talk he gave about it. [this video]
Approachable talk on Ng's state-of-the-art unsupervised learning results.
Indirectly, yes. The Coursera course teaches K-means clustering as an example of an unsupervised learning algorithm, along with a technique to work with larger data sets in a more manageable way (e.g. PCA to reduce dimensionality). With the basic building blocks from the ML course you have a much better understanding of what Google and Stanford did. Andrew Ng is also the course instructor for the Coursera ML course[1]. I am presuming that supplemental readings and Youtube videos by Prof. Ng (e.g. [this video] ) allows a sufficiently motivated individual to at least partially replicate the results in the OP. [1] Andrew Ng is also a co-founder of Coursera.
> By definition, a computer is a machine that processes and stores data as ones and zeroes. No, of course thats only true for digital computers. Otherwise, what would be the point of the headline? Not a great article but interesting news. I could imagine specialized computer for artificial neural networks (ANN) being commercially successful in the future. Not sure if this is what the people in this DARPA project are working on. As far as I can tell, there are a lot of breakthroughs in ANN's at the moment, especially in the realms of pattern (e.g. image) recognition: Ng: [this video] Hinton: [link]
Example of breakthroughs in neural networks for pattern recognition.
Video list and Hacker News comment labels from HackerTube (prompted by @karpathy), released under ODC-By 1.0, built on the open-index/hacker-news archive. Hacker News data is provided by Y Combinator; comments remain the work of their authors. Video titles, channel names and thumbnails belong to their creators and are shown via YouTube. HackerTube Watch is an unofficial edition. Data export of 2026-10-31.
Plays via YouTube's privacy-enhanced embed
11 Hacker News comments
From the HN archive; up to 20 kept per video, favoring different commenters. Recommendations first.
Read more
Citations, criticism and jokes
Read more