Towards the understanding of object manipulations by means of combining common sense rules and deep networks - Csákvári Máté (Conference of PhD Students in Computer Science, 2018)
Object detection on images and videos improved remarkably recently. However, state-of-theart methods still have considerable shortcomings: they require training data for each object class, are prone to occlusions and may have high false positive or false negative rates being prohibitive in diverse applications. We study a case that a) has a limited goal and works in a narrow context, b) includes common sense rules on ‘objectness’ and c) exploits state-of-the art deep detectors of different kinds. Our proposed method works on an image sequence from a stationary camera and detects objects that may be manipulated by actors in a scenario. The object types are not known to the system and we consider two actions: “taking an object from a table" and “putting an object onto the table". We quantitatively evaluate our method on manually annotated video segments and present precision and recall scores.