Convolutional Neural Network

Convolutional Neural Network 1) Explain with an example why the inputs in computer vision problems can get huge. Provide a solution to overcome this challenge. Consider a 500x500 pixel RGB image fed to a fully connected neural network for which the first hidden layer has just 1000 hidden units. For this image, the number of input features will be 500*500*3=750,000, i.e. the input vector will be 750,000 dimensional. The weight matrix at the first hidden layer will therefore be a 1000x750,000 dimensional matrix which is huge in size for both computations as well as storage. We can use convolution operation, which is the basis of convolutional neural networks, in order to address this challenge. 2) What are the features likely to be detected by the initial layers of a neural network used for Computer Vision? How is this different from what is detected by the later layers of the neural network? The earlier layers of the neural network detect simple features of an image, such as edges or corners. As we go deeper into the neural network, the features become increasingly complex, detecting shapes and patterns. The later layers of the neural network are capable of detecting complex patterns such as complete objects. 3) Consider a filter [-1 -1 -1; 0 0 0; 1 1 1] used for convolution.What edges will this filter extract from the input image? This filter will extract horizontal edges from the image. To get a more concrete understanding, consider a grayscale image represented by an array with the following pixel intensities: [0 0 0 0 0 0; 0 0 0 0 0 0; 0 0 0 0 0 0; 10 10 10 10 10 10; 10 10 10 10 10 10; ]. From the array, it should be apparent that the top half of the image is black, whereas the lower half is a lighter color forming an apparent edge at the center of the image. The convolution of the two will result in the array [0 0 0 0; 30 30 30 30; 30 30 30 30; 0 0 0 0;]. It can be observed from the values in the resultant array that the horizontal edge has been identified. 4) How do you address the issue of the edge pixels being used less than the central pixels during convolutional operation? In order to address the issue of the filter or kernel extracting information from the edge pixels less in comparison to the central pixel, we can use padding. Padding is essentially adding one or more additional rows or columns of pixels along the boundary of the image. The padding forms the new edge pixels of the image and therefore results in insufficient extraction of information from the original edge pixels. Padding provides the added advantage of preventing the shrinking of an image as a result of the convolution operations. 5) For a 10x10 image used with a 5x5 filter, what should the padding be in order to obtain a resultant image of the same size as the original image? For an image without padding, the size of the image nxn after convolution with a filter of size fxf is given as (n-f+1)x(n-f+1) (You can verify this with the example in question 3. of this section). Now, if you add padding of 1 pixel, in effect, this will be like adding a border of one pixel around an image, consequently increasing the length and breadth of the image by 2. That is, for a padding p, the size of the original image becomes (n+2p)x(n+2p). To maintain the size of the image after convolution, you will need to choose p=f-12. For the given problem, therefore, we will need to choose p=(5-1)/2 =2. 6) Given a 5x5 image with a 3x3 filter and a padding p=1, what will the size of the resultant image be if a convolutional stride of s= 2 is used? For an nxn image with an fxf, padding p, and stride length s, the size of the resultant image after convolution has the shape n+2p-fs+1 x n+2p-fs+1. Therefore for the given problem the size of the resultant image will be (((5+2*1-3)/2) +1) x (((5+2*1-3)/2) +1)= 3 x 3. 7) For an RGB image of dimensions 10x10x3 convolved with a 3x3 filter, what will be the size of the resultant image? The convolution operation is not possible for a 10x10x3 image with a 3x3 filter as the third dimension (or the number of channels) must be the same in order to achieve convolution. Alternatively, if a 10x10x3 image is convolved with a 3x3x3 filter, the dimensions of the resultant image will be 4x4. 8) What will be the change in the size of the results of the convolution is performed with a 1x1 filter on a 10x10 image with no padding? The size of the image will remain unchanged; that is, the height and width of the resultant image will be the same as the resultant image. However, the number of channels in the resultant can be changed with a 1x1 convolution, thus increasing or decreasing the dimensionality of the input. 9) In order to extract multiple features from an image, it is common practice to use a number of filters and then stack them up to form the resultant. So for a 10x10x3 image convoluted using ten filters of dimensions 5x5x3, how many parameters must be learned to form the filters alone? Each 5x5x3 filter has 5*5*3=75 features. Therefore for ten such filters, the total number of features will be 75*10=750 features. (NOTE: The bias has not been accounted for in this calculation. However, notice how this number is very small in comparison to the extremely large number of features mentioned in question 1 when convolution is not used.) 10) How many parameters are to be learned in the pooling layers? No parameters are to be learned in the pooling layers. In general, the pooling layer has a set of hyperparameters describing the filter size and the stride length, which are set and work as a fixed computation. 11) For max-pooling done on a 6x6x3 image with a filter of size f=2 and padding p=0 with stride s=1, what would be the size of the output image? The size of the resultant image will be as per the formula n+2p-fs+1. Additionally, the number of channels in the resultant image will be the same as that in the input image. Therefore, for the given problem, the dimensions of the resultant image will be (6+2*0-2+1) x (6+2*0-2+1) x 3 = 5 x 5 x 3. 12) Suggest a way to train a convolutional neural network when you have a quite small dataset. If you have a dataset that will not be enough to train a convolutional neural network sufficiently well ( and even otherwise sometimes in the interest of time or resource), it is suggested to use transfer learning to solve your machine learning problem. Depending on the size of the dataset and the budget of time and resources you have at your disposal, you can choose to train only the last classification layer or a few of the later layers. Alternatively, you could use transfer learning to train (when the size of the data set is large enough to allow this) all the layers with the parameters initialized from a previously trained model. Owing to the large availability of open-source pre-trained models, transfer learning for computer vision problems is both easy and strongly advised. 13) Explain why mirroring, random cropping, and shearing are some techniques that can help in a computer learning problem. There is often limited data available to solve computer vision problems, which just isn't enough to train the neural networks. Techniques like mirroring, random cropping, and shearing can help augment the existing dataset and create more training data from the existing data, thereby ameliorating the issue of limited training data. 14) Mention a method that can be used to evaluate an object localization model. How does it work? Intersection over Union (also known as IoU) is a commonly used method to evaluate the performance of the object localization model. The overlap between the ground truth bounding box and the predicted bounding box is checked, and the ratio of the intersection of the areas to the Union of the areas is calculated. If this ratio called IoU is found greater than some threshold (usually set to 0.5 or higher), the prediction of the model is considered correct. 15) How can IoU be used for resolving the issue of multiple detections of the same object? The IoU method also helps in the non-max suppression technique for eliminating multiple detections of the same object. It is common for object localization models to predict multiple bounding boxes for the same object. In such cases, the box with the maximum probability among the overlapping boxes of a class is taken. The IoU of this box with the other overlapping boxes is then checked. If this IoU is above a certain threshold, the boxes are considered to be detecting the same object, and the lower probability boxes are eliminated. 16) Mention a scenario that would require the use of anchor boxes. When detecting multiple classes of objects, there is a scenario where the center of two bounding boxes capturing objects of two different classes occurs on the same point or grid cell. Despite the overlap of the two boxes owing to the fact that both the boxes indicate different objects, they will both require to be retained (something a grid cell in the sliding window technique isn't ordinarily able to accomplish). For this purpose, a number of anchor boxes of different dimensions are used, and vectors similar to the original output vector of a grid cell are now provided for each anchor box in the new output vector. The best-fitting anchor box for a particular class of objects is used to indicate that the grid cell contains the center of the object, thus allowing for multiple overlapping object detection. 17) How does the Siamese Network help to address the one-shot learning problem? Siamese Network works by encoding a given image with a learned set of parameters such that the distance between the encoding is large when the image is of two different people, and the distance is small when the two images compared are of the same purpose. Training this neural network, therefore, involves learning parameters such that these conditions of the encoding are satisfied when provided with images of two different people or two images of the same person. General Interview Questions on Computer Vision 18) What purpose does grayscaling serve? Grayscaling helps to reduce the dimension of the image and thus allows for reduced computation time and effort. Further, it reduces the complexity of models and functions required for various operations. Some functions like edge and contour detection and machine learning problems Optical Character Recognition perform better or are implemented for working only with grayscale images. 19) What color to grayscale conversion algorithm does OpenCV employ? What is the logic behind this? The color to grayscale algorithm in OpenCV uses the formula Y=0.299*R+0.587*G+0.114*B. This makes it similar to the luminosity method, which averages the color intensity values weighting them in accordance to human perception of different colors, i.e., it accounts for the fact that humans perceive green more strongly than red, and red more strongly than blue, which is apparent from the weightage given to each color's pixel intensity. Additionally, the OpenCV grayscaling algorithm takes into consideration the nonlinear operation used to encode images. 20) What is translational equivariance? What brings about this property in Convolutional Neural Networks? Translational Equivariance is a property where the position of an object in an image will not affect its detection in the image. That is, if an image is shifted a few pixels to the right or to the left, the output will merely change its position equally and otherwise remain unaffected. Translational Equivariance is an important property of Convolutional neural networks and is brought about by the parameter sharing concept. 21) What is the basis of the popular EAST text detector? EAST text detector is based on a fully convolutional neural network adapted for text detection. It has gained immense popuarity because of its text detection accuracy in natural scene images. 22) What is the basis of the state-of-the-art object detection algorithm YOLO? YOLO is an object detection algorithm that is based on a Convolutional Neural Network and capable of working in real-time. It provides accurate detections for a large variety of objects and can be used as a solution or a starting point for transfer learning for many computer vision problems. 23) Can you name some of the different types of image filters used in OpenCV? Gaussian Blur, Bilateral Filter, Gabor Kernels, Median Blur, Dilate, Box Filter, and Erode. Computer Vision Interview Questions Asked at Facebook 24) Write the code to handle the placement of Tetris Blocks in a Tetris game. 25) What do you understand by Bundle Adjustment? 1. What is computer vision ? 2. What are languages supported by Computer vision ? 3. What are computer vision libraries ? 4. How many algorithms in opencv ? 5. What are machine learning algorithms available in opencv ? 6. How many types of image filters in OpenCV ? 7. How many types of video filters in OpenCV ? 8. What Is Dynamic Range? 9. What Do You Meant By Color Model? 10. What Are The Types Of Light Receptors? 11. What Is Chromatic Adoption? 12. What Is Meant By Pixel? 13. What Is Geometric Transformation? 14. What Is Meant By Mach Band Effect? 15. Define Digital Image? 16. What Do You Meant By Gray Level? 17. What Is Meant By Path? 18. Give The Formula For Calculating D4 And D8 Distance. 19. Explain What Is Simultaneous Contrast? 20. In what way does Cones and Rods Are Distributed in Retina? 21. Define Brightness? 22. What Is Meant By Machband Effect? 23. Define Sampling And Quantization? 24. What Are Sampling And Quantization? 25. Differentiate Photopic And Scotopic Vision? 26. Define Resolutions? 27. What Is Image Translation And Scaling? 28. What Do You Meant By Shrinking Of Digital Images? 29. Explain Subjective Brightness And Brightness Adaptation? 30. What Are The Steps Involved In Dip? 31. Do You Know What Rectification Is In Image Processing? 32. What Is Meant By Illumination And Reflectance? 33. List The Categories Of Digital Storage? 34. What Are The Differences Between Structural Patterns & Morphological Structural Element? 35. Write Short Notes On Neighbours Of A Pixel? 36. Specify The Elements Of Dip System? 37. What Is Hue And Saturation? 38. What Is Luminance? 39. What Is Simultaneous Contrast? 40. What Is Recognition And Interpretation? 41. How can we Increase the Quality of an Image? 42. What Is Image Transform? 43. What Do You Meant By Zooming Of Digital Images? 44. What Is The Need For Transform? 45. List The Hardware Oriented Color Models? 46. Explain What Are The Differences Between The Books Digital Image Processing And Digital Image Processing?