How is the world of photography changing with the advent of artificial intelligence? Will the profession of the photographer still exist? After this portrait robot perhaps more than someone might get worried
Machine learning and artificial intelligence stand revolutionizing the world of photography, especially in the smartphone sector where even inexperienced users want to be able to obtain satisfactory results simply by pressing their finger on the screen. And we really have i seen i miracles of algorithms because sensor technology is not so rapid in its evolution and in any case has a certain limit imposed by the size of the sensors.
Most of the algorithms apply in pre-shooting phase, to predict the optimal shooting parameters, and especially in post production phase to process the final file and get the most out of it. But some researchers have gone even further, attempting to create a portrait robot and completely automating the image capture process.
Smartphones are the main promoters of the automation of shots
The portrait robot: artificial intelligence and photography meet once again
Millions of images are taken nowadays probably, but making a quality photograph is certainly not an easy task. In fact, not only the shooting parameters and post production are important, but also everything the design process leading to the final shot: the choice of location, light control, camera angles and shots. Although photography as an art form has been the prerogative of humans until now, some researchers have tried to design robots that can simulate the performance of a real-life photographer.
With this in mind, researchers at Purdue University and Adobe have recently developed a new set of tools that allow a robot to take photographs fully automatically to humans, especially studio portraits. This framework, presented in an article published on arXiv, allows robots to follow a human user to an ideal point of the shot and then to proceed with the shot. The name of this new artificial intelligence system is LeRop and it was specifically designed to take pictures of a human being in indoor locations: a fully-fledged portrait robot in practice.
The robot designed by the researchers
How does LeRop work?
First of all LeRop guide the robot to a particularly interesting or favorable part of the location to take the picture. Then use a model for image evaluation to find an optimal view of the scene and finally a model of deep reinforcement learning (DRL) it is used to adjust the position of the robot and its orientation to ensure the best possible lighting condition. The researchers write in their article:
When the composition phase is activated, the robot attempts to adjust its position to form the view that best fits the given model image and finally takes a photograph. The model image used for framing calibration can be dynamically predicted using a photographic evaluation model standardized by the framework or manually selected from a user-predefined set.
LeRop is a interactive artificial intelligence system, a user can program it to follow a certain subject to the place where he deems it most appropriate to take the image, based on the indications provided. Once the set point is reached, the portrait robot begins to look for the best angle to get the most out of the light of the place. LeRop’s deep reinforcement learning (DRL) component is the one that definitively allows you to find the most suitable resume point, based on the template selected initially.
The researchers decided to equip the robot with one 360 degree video camera and a high-quality main camera, so you can have a complete view of your surroundings at any time without continuously rotating or passing through various viewpoints. Interestingly, the has a modular structure, which means that all its models can be replaced or adapted according to the needs of a certain user. The team tested LeRop in numerous field trials on three indoor scenes, integrating it into a simple robot called the Turtlebot. The robot took 20 photographs of a human user in each of these three locations, 10 with a predefined model and 10 using dynamically generated models. The average number of adjustments made by the robot was 11.20 for predefined models and 12.76 for dynamically generated ones. Furthermore, using the researchers’ artificial intelligence system, the robot could take a picture about 22.11 seconds using predefined templates and 22.40 seconds using dynamically generated ones.
Example of a shot made with LeRop
Robotic photography
The LeRop framework could be the first step towards creating an efficient robotic photography system capable of producing high quality portraits quickly and automatically. However, the system developed by the researchers still presents a number of limitations that could prevent its large-scale adoption. For example, so far, it has limited built-in computing power and therefore only works on a powerful remote computer. What’s more, researchers have so far only integrated it into Turtlebot, which is a simple robot with very few degrees of freedom. In their next jobs, they would like to test their system on a more complex robot with multiple degrees of freedom. The researchers conclude by saying:
Our system currently only supports a single person portrait. The new policies should be retrained to get better support for taking group photos. In the future, we would also like to test different aesthetic models of photographic evaluation and extend the work to outdoor scenes.
Artificial intelligence: the end of photographers?
Absolutely not. LeRop it cannot replace a human photographer (for now at least). We are not just talking about technical limits such as computing power or the limited number of scenarios in which the artificial intelligence system can be applied. There are also deeper motivations. First of all the photographer intends tell a story or an emotion with his shots, he gets in tune with the environment and with the subject. It’s not just about choosing the right light and angle; or rather it is a question of choosing the right light and angle to enhance one’s story and one’s vision of reality. This one machine is nowhere near capable of doing it.
Adobe Sensei, Adobe’s artificial intelligence
What is the portrait robot for and why is Adobe investing resources in it? It might be a valid “assistant” in the near future, guided by the indications of an experienced photographer. However, an application closer in time could be that of a software capable of give live suggestions photographers while shooting to optimize their photographs. What are you saying? Could it be useful to you? From the photography section that’s all! Keep following us for many other news and insights!