Notes about Cutting-Edge Technologies and Everything

Over the last decade, deep learning models have proven highly effective at performing a wide variety of machine learning tasks in vision, speech, and language. At Uber we are using these models for a variety of tasks, including customer support, object detection, improving maps, streamlining chat communications, forecasting, and preventing fraud. Many open source libraries, including TensorFlow, PyTorch, CNTK, MXNET, and Chainer, among others, have implemented the building blocks needed to build such models, allowing for faster and less error-prone development. This, in turn, has propelled the adoption of such models both by the machine learning research community and by industry practitioners, resulting in fast progress in both architecture design and industrial solutions. At Uber AI, we decided to avoid reinventing the wheel and to develop packages built on top of the strong foundations open source libraries provide. To this end, in 2017 we released Pyro, a deep probabilistic program...

Comparing Top Deep Learning Frameworks: Deeplearning4j, PyTorch, TensorFlow, Caffe, Keras, MxNet, Gluon & CNTK Skymind bundles Deeplearning4j and Python deep learning libraries such as Tensorflow and Keras (using a managed Conda environment) in the Skymind Intelligence Layer (SKIL), which offers ETL, training and one-click deployment on a managed GPU cluster. The SKIL Community Edition is free and downloadable here . Eclipse Deeplearning4j is distinguished from other frameworks in its API languages, intent and integrations. DL4J is a JVM-based, industry-focused, commercially supported, distributed deep-learning framework that solves problems involving massive amounts of data in a reasonable amount of time. It integrates with Kafka, Hadoop and Spark using an arbitrary number of GPUs or CPUs , and it has a number you can call if anything breaks. DL4J is portable and platform neutral, rather than being optimized on a specific cloud service such as AWS, Azure or Goog...

We have entered the data age, where insights from streams of data can offer organisations valuable insights. Obtaining these insights is done using machine learning and the next generation in machine learning, deep learning, offers businesses the ability to streamline operational processes and cut costs. As long as development costs remain under control, organisations can achieve a positive ROI on their investments.  The next phase of machine learning, deep learning, relies on large amounts of data to train the algorithms. Normally, this would incur high-costs, but thanks to IBM’s new Watson Studio costs can be kept low, resulting in a positive ROI for organisations. IBM Watson Studio allows you to train your models and easily add functionality thanks to their APIs using a pay-as-you-go model. IBM Watson Studio solves the main issues of developing neural networks: complexity, standardisation and skills gaps, making deep learning also available to small organisations. ...

There is a powerful technique that is winning Kaggle competitions and is widely used at Google (according to Jeff Dean), Pinterest, and Instacart, yet that many people don’t even realize is possible: the use of deep learning for tabular data, and in particular, the creation of embeddings for categorical variables. Despite what you may have heard, you can use deep learning for the type of data you might keep in a SQL database, a Pandas DataFrame, or an Excel spreadsheet (including time-series data). I will refer to this as tabular data , although it can also be known as relational data , structured data , or other terms (see my twitter poll and comments for more discussion).    From the Pinterest blog post 'Applying deep learning to Related Pins' Tabular data is the most commonly used type of data in industry, but deep learning on tabular data receives far less attention than deep learning for computer vision and natural language processing.  Details ...

We’re stuck.  There hasn’t been a major breakthrough in algorithms in the last year.  Here’s a survey of the leading contenders for that next major advancement. Details: https://www.datasciencecentral.com/profiles/blogs/what-comes-after-deep-learning

The hype around data science and machine learning has increased from already high levels in the past year. Data and analytics leaders should use this Hype Cycle to understand technologies generating excitement and inflated expectations, as well as significant movements in adoption and maturity. The Hype Cycle The Peak of Inflated Expectations is crowded and the Trough of Disillusionment remains sparse, though several highly hyped technologies are beginning to hear the first disillusioned rumblings from the market. In general, the faster a technology moves from the innovation trigger to the peak, the faster the technology moves into the trough as organizations quickly see it as just another passing fad. This Hype Cycle is especially relevant to data and analytics leaders, chief data officers, and heads of data science teams who are implementing machine-learning programs and looking to understand the next-generation innovations. Technology provider product marketers and strategists...

The resource below describes the following frameworks: TensorFlow  Theano Caffe MXNet Apache SystemML (incubator project)  BigDL  DistBelief Details >>>

Comparing Top Deep Learning Frameworks: Deeplearning4j, PyTorch, TensorFlow, Caffe, Keras, MxNet, Gluon & CNTK:  https://deeplearning4j.org/compare-dl4j-tensorflow-pytorch

How to build a multilayered LSTM network to infer stock market sentiment from social conversation using TensorFlow: https://www.oreilly.com/ideas/introduction-to-lstms-with-tensorflow

(by Andrej Karpathy, Director of AI at Tesla) Neural networks are not just another classifier, they represent the beginning of a fundamental shift in how we write software. They are Software 2.0. The “classical stack” of Software 1.0 is what we’re all familiar with — it is written in languages such as Python, C++, etc. It consists of explicit instructions to the computer written by a programmer. By writing each line of code, the programmer is identifying a specific point in program space with some desirable behavior. In contrast, Software 2.0 is written in neural network weights. No human is involved in writing this code because there are a lot of weights (typical networks might have millions), and coding directly in weights is kind of hard (I tried). Instead, we specify some constraints on the behavior of a desirable program (e.g., a dataset of input output pairs of examples) and use the computational resources at our disposal to search the program space for a pr...

An overview of the main existing recommendation system algorithms: Collaborative filtering Matrix decomposition Clustering Deep learning approach Details:  https://blog.statsbot.co/recommendation-system-algorithms-ba67f39ac9a3