I suspect that less than 10 years from now, all of the DL training/architecture tricks that came from the arXiv firehose over 2015-2019 will have been entirely superseded by automated search techniques. The future: no alchemy, just clean APIs, and quite a bit of compute.

Jan 7, 2019 · 6:44 PM UTC

17
131
11
602
High-level APIs, I should add. In the future, doing tensor-level manipulations will feel like writing assembly.
5
20
1
199
Replying to @fchollet
We are pushing learning more on the meta-level (so learning to learn to learn …) but the fundamental challenges will probably be inherently the same: priors, overfitting, …
Replying to @fchollet
Compute is cool and all, but what about some good old fashioned understanding to put the tricks in perspective and provide theory-driven advances?
4
6
83
Replying to @fchollet
Agreed. But I don’t think the automated search techniques will work without any priors. The tricks from arXiv will make up the priors to make the search more efficient.
4
3
45
Replying to @fchollet
I see a divergence. The training for autonomous systems will be different from the training for high computational scenarios. Adaptive reasoning is different enough from systematic reasoning.
Replying to @fchollet
do you plan to evolve keras in that direction? or would keras be at assembly layer.
Replying to @fchollet
Abd the USPTO. Have you ever done a patent Search? Yikes!
Replying to @fchollet
We will put all the magic into automated search techinques. I can image that in the near future there will be neural automated search, which was trained on results of all arXiv papers.
1
5
Replying to @fchollet
Really a one size fits all thing? Maybe, but then I'd expect the occasional guru to succeed and modify such an automation result to something several orders of magnitude better and this for another 20 years.
Replying to @fchollet
Shor's algorithm can find the parameters for Neural Networks. You just need more powerful quantum computers...
1
3
Load more