Machine Learning

  • Planning and Scheduling Using Temporal Actions Project Summary

    Project’s main objective was to develop a methodology for solving planning problems involving time by using simple planning tools. During the project I studied the field of classical and temporal planning, learned the PDDL language which is used for describing planning problems and designed a method for solving temporal planning problems using classical planning solver written in python (Pyperplan).
    Categories:
    Tags:

  • Inverse Reinforcement Learning in Contextual MDPs

    In this project, we consider the Inverse Reinforcement Learning problem in Contextual Markov Decision Processes. Here, the reward of the environment depends on a hidden static parameter referred to as the context, i.e., each context defines an MDP. The agent does not observe the reward, but instead, is provided with expert demonstrations for different contexts. The goal of the agent is to learn a mapping from contexts to rewards that...
    Categories:
    Tags:

  • CNN Spatial Optimizations

    Convolutional neural networks (CNNs) compute their output using weighted-sums of adjacent input elements. This method enables CNNs to achieve state-of-the-art results in a wide range of applications such as computer vision and speech recognition. However, it also comes with the cost of high computational intensity. Shomron et al purposed exploiting the spatial correlation inherent in CNNs and predict activation values, thus reducing the needed computations in the network. They introduced...
    Categories:

  • Arbitrage system for cryptocurrency trading

    Cryptocurrency is still a new and inefficient market. Several Cryptocurrency exchanges exist around the world and the bid/ask prices they propose can be briefly different from an exchange to another. The purpose of our arbitrage system is to automatically profit from these temporary price differences while being market-neutral. System features: In-Exchange and Cross-Exchange arbitrage opportunities founder. General and Scalable system to add new Exchanges and coins, set limits on arbitraging...
    Categories:

  • L32- Adversarial Attacks on Navigation Systems

    Visual navigation systems have become essential in nowadays autonomous systems, with deep learning algorithms being the state-of-the-art. In this project, we will work towards developing adversarial attacks on such algorithms. We aim for generating passive example attacks that, when injected to the scene being observed by the visual sensor, will cause the navigation algorithm to diverge from its trajectory. We will start from attacking a white-box algorithm and examine several...
    Categories:
    Tags: |

  • L26- Perceptually aligned gradients

    Perceptually aligned gradients is a known phenomenon where strong adversarial attacks (epsilon=1) produce samples which greatly resemble samples from different classes. In this project, we wish to explore several questions, such as: When does this phenomenon reproduce? and why? Does it depend on different attacks and norm limitations? We wish to study the above questions, especially interesting are EPGD attacks as described in the second link. https://arxiv.org/pdf/1910.08640.pdf https://arxiv.org/pdf/1911.07198.pdf
    Categories:
    Tags: |

  • Action elimination DQN: exploration via confidence bounds

    Background: Deep reinforcement learning (DRL) methods such as the Deep Q-Network (DQN) have achieved state-of-the-art results in a variety of challenging, high-dimensional domains. One of the major problems in DRL is that the state and action spaces are large, and the task often becomes an exploration problem. While DRL methods are new, RL is a well established field. There exist various algorithms in the tabular case which ensure proper exploration and convergence to the...
    Categories:
    Tags:

  • Implicit bias in optimization algorithms

    On many ML problems (most notably DL’s neural networks), different optimization algorithms lead to different results. This is known as implicit bias of the optimization algorithm. Soudry et al. have shown that GD algorithm, when operating on Linearly separable data, given enough time, leads to the L2 Max-Margin solution obtained by SVM. In our project, we empirically test different optimization algorithms performance on Linearly separable data. We analyze impact of...
    Categories:

  • Ultrasound Nerve Segmentation

    Deep learning in general, and segmentation problems in particular, are the subject of much research done in the field of learning systems. The rising power of computers enables the implementation of complex networks containing thousands of variables in order to create optimal architecture for such problems. However, there are several approaches to dealing with segmentation problems, each of which has its own advantages and disadvantages. This project deals with a...
    Categories:
    Tags:

  • L22- Learning Ensemble Grasp Predictors for Robotic Manipulation

    Robots are rapidly becoming central tools to many industries, from manufacturing through e-commerce and autonomous vehicles to medicine and healthcare. A major challenge to current robotic manipulators is grasping objects, especially when the object shapes or poses are unknown in advance. Recent advances in Deep Learning have allowed training powerful grasp predictors, suggesting where a robot should attempt to grasp an object. In this project, we will train an ensemble...
    Categories:
    Tags: |

  • IMPROVING INFERENCE ALGORITHMS FOR EXTREME CLASSIFICATION

    Extreme classification is a growing research area dealing with multi-class tasks with an extremely large number of classes. One new challenge arising in this setting is to perform inference in a reasonable time, namely quicker than common-practice multi-class algorithms which require linear time in the number of classes. Many recent studies in this area learn hierarchical classification models, which allow logarithmic inference time. In the project we propose a different...
    Categories:
    Tags:

  • L24- Starcraft2 Macro Bot

    Starcraft2 is a complex game where two (or more) players compete to collect resources, build armies, and overcome the opponent to win the game. One of the major complexities in Starcraft compared to other strategy games is the real-time aspect; units and building must be accurately controlled live on top of executing high-level strategies. Therefore, one promising direction is to utilize deep learning. Neural nets can encode previous experience in...
    Categories:
    Tags: |

  • Unsupervised Domain Adaptation for Human Movement

    We present a novel and effective approach for video to video translation through generative adversarial networks. Given an input video of a person and a target video, our model generates a video of that same person mimicking the motion of the person in the target video. Our model can be used for various tasks, from ”teaching” someone to dance like a skilled dancer to the generation of realistic videos of...
    Categories:
    Tags:

  • Successor Features for Minecraft

    Transfer Learning in the field of machine learning is the method of using knowledge gained while learning previous tasks on a new task. In Reinforcement Learning, one way to utilize Transfer Learning is using the knowledge gained while training an agent in a specific domain on a task with its respective reward function, on a new task in the same domain but with different reward function. Barreto et al. (2017)...
    Categories:

  • Multi-Agent Deep Reinforcement Learning (MARL) in StarCraft 2

    Abstract This document summarizes the efforts we made training an AI to play the popular PC game “Starcraft 2”, using Multi Agent deep reinforcement learning methods. As part of our solution we have trained several variants of agents, using Multi agent methods to complete the game, and doing so, have managed to get better results than the baseline by about 25%. Our agents were trained using different Reinforcement Learning algorithms,...
    Categories:

  • Cryptocurrency Portfolio Management: The Deep Reinforcement Learning Approach

    Portfolio Management is the decision-making process of allocating wealth across a set of assets, it is a fundamental problem in computational finance and has been extensively studied across several research communities. Cryptocurrencies are digital assets designed to work as a medium of exchange that uses strong cryptography to secure financial transactions, control the creation of additional units, and verify the transfer of assets. In this work, we try to adapt...
    Categories:

  • Algo-Trading with RNNs

    Stock markets today have become fast moving dealing with a very high daily trade volume. It makes it difficult to be able to make reliable investments as prices are very volatile and can change rapidly. Inspired by recent advances in deep learning and the growing interest in RNN networks, we aim to use various deep learning techniques to implements models able to make reliable and accurate stock market predictions. We...
    Categories:

  • Bayesian Online Training of Neural Networks

    We view the neural network as a probabilistic model. In this project, we propose an online Variational Bayes approximation. In an online approach, the data points are received one at a time, and the parameters of the neural network are updated after each observation. The Bayesian framework makes an online approach to learning which is natural to implement using Bayes’ rule. The assumption of a small change in the parameters...
    Categories:

  • Speech Recognition/Classification using Deep Neural Networks

    While todays interaction between man and machine is done many through screens a much more natural way to do so is via speech. Many industry leaders are already developing and improving algorithms that are meant to mimic and understand human language. Achieving human like performance will open a completely new way to interact with devices and offer new applications we haven’t thought of before. In this Project we test state...
    Categories:

  • Online Distillation Network

    We propose a lifelong learning system that utilizes knowledge obtained from previously learned skills in order to help generalize and accelerate the learning of new skills. A skill is a strategy learned by the agent in order to perform a certain task. A Multi-Task Agent is an agent that can use multiple skills. Policy Distillation is an offline learning method to create Multi-Task Agents, by using multiple copies of the...
    Categories:

  • L20- Deep Learning Scene Generalization

    במסגרת הפרויקט הסטודנטים יתכננו ויממשו מערכת לומדת אשר תתרגם קלט (תמונות/ענן-חלקיקים) לאוסף צורות פשוטות. תחילה, הרשת תבצע פירוק היררכי של האובייקט לתתי חלקים ומשם תפיק קירוב גיאומטרי לטובת ייצוג דחוס ותמציתי של הסצנה.
    Categories:
    Tags: |

  • L19- משחקי תוקפים ומגנים מבוססי למידה חישובית

    הפרויקט עוסק בבעיית אסטרטגיות תקיפה של קבוצת תוקפים של קבוצת מטרות, בנוכחות קבוצת מגנים, ומבוססת על עקרונות מתקדמים של למידת חיזוק של קבוצות. דוגמאות לכך הינם משחק אטארי .. בו קבוצת טילים תוקפת מטרות קרקע בנוכחות קבוצה של טילי מגן. דוגמא אחרת ואולי יותר מעניינת מדברת על קבוצה של שחקנים שצריכה להעביר קבוצה של כדורים לצד שני של המגרש בנוכחות קבוצה של מגנים (סוג של פוטבול אמריקאי..). עקרון התכנון והבקרה...
    Categories:
    Tags: |

  • Multi-task DQN

    The success of multi-task deep reinforcement learning has been limited so far and the community is currently lacking explanations. We suspect that the main reasons are instabilities between gradients that are coming from different tasks. We support these claims with results from representation learning, a powerful tool to discover properties of learning algorithms. In particular, t-SNE has been demonstrated to be useful for visualizing the learned representation of Deep Reinforcement...
    Categories:

  • Solving Text-Based Games with Reinforcement Learning

    Text-based games are rich virtual environments with textual interactions using mostly natural language. Since the game states are unobservable, these games are especially challenging for automatic game players. In our project, we attempt to solve a virtual sub-domain by learning corresponding control policies. We employ a deep reinforcement learning framework to jointly learn state representations and action representations, using game rewards as feedback. We approximate the Q-function using an interaction...
    Categories:

  • Learning to Play RoboCup Soccer from Scratch using Deep Learning

    The novel approach of Deep Deterministic Policy Gradient (DDPG) presented in [\cite{lillicrap2015continuous}] has shown great success in tackling Reinforcement Learning (RL) problems featuring continuous state and action spaces. [\cite{hausknecht2015deep}] extended the DDPG algorithm for solving multiple parameterized continuous action space implemented in the Half-Field-Offense (HFO) RoboCup environment. In this paper we propose an alternative architecture to solve the HFO RL problem in terms of network size and training time while...
    Categories:

  • Control and Deep Learning in Dynamic Systems

    The objective of this project is to find a generic method for solving non-linear, continuous control problems, using deep learning and an Iterative Linear Quadratic Regulator controller. During the course of this project, we solved a private control problem, for which we built an algorithm that may be applied to generic control problems. In the private problem, we are given a surface, centered on which is a robotic arm with...
    Categories:

  • Transfer Learning in DQN using weighted layers copying

    In DRL, we can refer to transfer learning as the ability to use knowledge gained while training an agent in one domain and applying it to the training of another agent, usually in a different domain. By transferring the weights of different parts of the networks, we sought to improve the learning rate and maximum reward achieved by the DQN algorithm.
    Categories:

  • Monte-Carlo Tree Search in real-time problems and in Machine Learning

    Monte-Carlo Tree Search is a heuristic search algorithm used mainly for decision making processes. In this project, we explored the use of Monte-Carlo Tree Search in many challenging environments, both as a real-time agent, and in the learning phase. How will the vanilla algorithm contend with OpenAI Gym's challenging environments? What heuristics and optimizations can we apply to bolster the performance? Through testing, analysis and inspiration from academic papers, we...
    Categories:

  • Analysis of Ultrasonic Vocalization of Rodents

    Autism spectrum disorders (ASD) are a range of complex neurodevelopmental disorders that mainly affect behavior and cognition. It is diagnosed by symptoms in two core behavioral dimensions: persistent deficits in social communication and social interaction, and restrictive, repetitive and stereotyped patterns of behavior. ASD is known to be a multifactorial disorder; Cumulative evidence point to environmental and epigenetic changes as the major causes of autism while during the last decade...
    Categories:

  • Feature Extraction Approach in Face Recognition

    In recent years, facial recognition systems, which are based on classifying neural networks, managed to achieve extraordinary high performances, often exceeding those of humans. However, vital drawbacks of these systems prevent them from executing significant tasks, such as recognizing a face of a person without designated training on his images, enlarging the pool of subjects the system is able to recognize, or even determine a face doesn’t belong to the...
    Categories:

  • L17- Learning Black Jack Using RL

    In this project we will learn to get the optimal playing policy in black jack using reinforcement learning. The project will require developing a simulator of the environment and running different RL algorithms such as Q-learning, UCB and DQN. Final results will be showing the ability to learn known results such as basic strategy and cards counting and in the latter stage results for some open questions in the field...
    Categories:
    Tags: |

  • L16- Projection-based Generative Models

    Recent advancements in deep generative modeling, and especially Generative Adversarial Networks (GANs), propose to replace the original input data with features from  a pre-trained network to allow a more stable training procedure and working in a low-data regime. The recent ProjectedGAN paper exhibited great results in generating new images when there is very low data available (e.g., images of Pokemons). In this project, we will investigate ways to improve this...
    Categories:
    Tags: |

  • L15- Finding Good Discrete Priors for VAEs

    Recent advancements in deep generative modeling replace the continuous latent vector representation with discrete latent representations and show improved performance in generation of new data. Most of these models, such as VQ-VAQ, VQ-GAN and DreamerV2 (RL), are based on Variational Autoencoders (VAEs). In the VAE framework, the user must choose a prior that generates latent variables, and in the continuous version, a simple Gaussian prior is usually chosen. However, in...
    Categories:
    Tags: |

  • L14- Bootstrap Your Own Generative Model (BYOGM)

    Recent advancements in deep generative modeling replace the loss function with features extracted from pre-trained models for better generative capabilities. This is true for the various types of generative models, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs). In this project, we will investigate the need for these pre-trained networks and try to replace them with a bootstrapped version of our generative model. The project may involve different...
    Categories:
    Tags: |

  • L13- עקיבת וידאו אחר מטרות מתמרנות בעזרת רשתות נוירונים CNN RNN

    במסגרת הפרוייקט הסטודנטים יצלמו תרחיש של מטרות נעות עם הסתרות. לדוגמא: אנשים הולכים / רכבים נוסעים/ שחיינים בים/  בעלי חיים נעים האובייקטים הנבחרים לעקיבת מחשב מדי פעם מוסתרים . יש לפתח עוקב שיבצע עקיבה אחר המטרות הנבחרות על ידי המשתמש תוך התמודדות עם הסתרות. העוקב יבוסס על רשתות עמוקות שעברו לימוד לזיהוי אנשים/רכבים/אובייקטים נבחרים עיבוד תמונה פרדיקטורים ולוגיקה מתאימה. הקלט לרשתות עמוקות אלה הינו התרחיש המצולם והפלט מהאלגוריתם הינו עקיבת...
    Categories:
    Tags: |

  • L12- (RL) Visualizing Domain Attributes

    RL problems are canonically implemented as a Markov Decision Process. The MDP defines the dynamics of the environment. Some of these dynamics are easy to see and understand while others less so. Visualizing certain attributes of the  domain can significantly impact user’s ability to understand and interpret an agent’s behavior. This project will aim to provide visualizations for some of the following attributes: 1.Agent Rewards – Built in rewards of...
    Categories:
    Tags: |

  • L11- Active Perception on ImageNet with RL

    The goal of active perception is to optimize confidence and cost by utilizing sensors intelligently. We aim to develop a RL-based agent that scans parts of an image, sends the output to a predictive model, and, based on the prediction and confidence of the predictive model, decides if more parts of the image need to be scanned
    Categories:
    Tags: |

  • L09- תכנון ולמידה בסביבה רציפה מורכבת Planning and learning in continues factored environments

    העולם סביבנו הינו גם רציף וגם בדיד – אנו יודעים את סדר הפעולות שאנו רוצים לעשות, להרים כדור, להניח כדור וכו׳, ובד בבד אנו מתכננים בעולם הרציף כיצד לבצע דברים – היכן להניח את הכדור, כמה מים למלא במיכל, לאיזו טמפרטורה לחמם את החדר וכו׳. בנוסף העולם סביבנו מורכב מתתי בעיות שעובדות במקביל, ולעיתים משפיעות אחת על השנייה ולעיתים לא – תזוזה של כיסא בחדר אחד משנה את מצב אותו...
    Categories:
    Tags: |

  • L08- תכנון והוצאה לפועל של לוח זמני רכבות Planning and control of trains schedule

    לוח זמני רכבות הינו דבר אשר נקבע מראש על סמך מרחקים בין תחנות, מהירויות נסיעה, וכן השערות לגבי הגעת נוסעים לתחנות. בפועל המציאות שונה מהתכנון, מגיעים נוסעים בכמות שונה, ולכן זמן עצירה בתחנה הינו שונה מהתכנון. בנוסף יכולות להיות תקלות ורכבות עלולות להתקע ולקלקל את זמני ההגעה של הרכבות, כפי שכולם בוודאי חוו על בשרם. בפרויקט זה נמדל את התלויות בין רכבות שונות ותחנות שונות במרחב מפורק (factored representation) ונעשה...
    Categories:
    Tags: |

  • L07- תכנון וניהול אופטימלי של תיק מניות ממונף Management of Leveraged stock portfolio

    תיק מניות הינו דבר אשר מצריך התערבות ואיזון שוטפים, כתלות בסוג התיק ובסיכון כך גם תדירות ההתערבות. בפרויקט זה נמדל תיק השקעות אשר מחזיק מדדים בסיכון גבוה, כך שנדרשת התערבות אך מכיוון ומדובר במדדים, תדירות ההתערבות לא ברורה (זה איננו מסחר יומי, ומצד שני לא איזון רבעוני). סוגיה נוספת הינה שככל שנעשות יותר קניות מבנה התיק משתנה (Lots), ולכן קבלת ההחלטות יכולה להיות שונה – מבנה מצב שונה ופעולות שונות....
    Categories:
    Tags: |

  • L05- בקרה של רשת תחבורה / Control of transportation networks

    רשת תחבורה הינה מודל המתאר את הזרימה של מכוניות בצמתים מרומזרים וכבשים בערים. בעולם של מרכזי ערים צפופים וריבוי מכוניות על הכבישים, הצורך לנתב בצורה חכמה רכבים ולמנוע פקקים הולך וגובר. בפרויקט זה נמדל רשת רמזורים, ונבקרה בצורה מבוזרת על מנת להיות מסוגלים להתמודד עם טופולוגיות שונות (גודל חיבוריות). הגישה בפרוייקט זה תהיה על סמך כלים של Multi agent reinforcement learning אשר מאפשרת לכל צומת להיות עצמאית כסוכן ברשת, אך...
    Categories:
    Tags: |

  • L06- תכנון משימה על ידי כלים של למידה עמוקה לא מונחית Task planning with deep unsupervised learning

    תחום הבינה המלאכותית מכיל מספר תתי תחומים אשר כל אחד מהם טוען ליכולת להגיע לבינה מלאכותית כללית, אך כל אחד מהם נתקל בקשיים אופייניים. תחום תכנון המשימה, הינו תחום הכולל חישוב משימה מתחילה ועד סופה על כל רבדיה. לדוגמא שתי מכוניות אוטונומיות הצריכות לאסוף 3 נוסעים שונים ממיקומי התחלה שונים ליעדים שונים צריכות לדעת מי אוספת את מי – החלטה דיסקרטית; מתי – החלטה רציפה; סדר פעולות (רכב אחד אוסף...
    Categories:
    Tags: |

  • L02- למידה של מידע זמני (סדרות זמן) תחת מגבלות של מידע חסר ברפואה

    Time series(TS) is one of the most common types of data, specifically, TS analyses play an important role in medical research. Unfortunately, time series in the medical domain tend to be missing and sparse, often leading to wrong results. While most algorithms take this into account, specific types of missing mechanisms, which are very common in medical data, are more difficult to attend to. In this project, we aim to...
    Categories:
    Tags: |

  • L01-רפואה מותאמת אישית מתוך גרפי אינפורמציה הדדית

    גרפי אינפורמציה הדדית (mutual information) מייצגים את הקשר בין גדלים ביולוגיים שונים, לרוב גרפים אלו נלמדים מתוך המידע הנאסף מניסוי בודד וללא שימוש במידע חיצוני. גרפים אלו מהווים את אחת מאבני היסוד במחקר רפואי. האתגר בלמידת קשרים מתוך ניסוי בודד הוא שלשם בניית הקשר אנחנו מאבדים את האינפורמציה על הדגימה הבודדת(או החולה הבודד), ובכך מאבדים את היכולת לקבל מסקנות מותאמות לכל מטופל. בפרויקט זה נציע וננתח שיטות לחילוץ מסקנות ברמת...
    Categories:
    Tags: |