Project 5: Lights, Camera

Github Classroom assignment

git clone --recurse-submodules <repo-url>

git submodule update --init --recursive

You can find the algo for this project here.

Introduction

In the Ray assignments, you implemented a ray tracer that projects a 3-dimensional scene on a 2-dimensional plane. Ray tracing, as you probably have experienced, can be very slow.

In this assignment, you will design a real-time scene viewer using components from previous labs including Parsing, Transformations, Trimeshes, VAOs, and Shaders.

Requirements

Parsing the scene

Similarly to Ray, you will use the same scene parser from Lab 5: Parsing to read in scenefiles. You are expected to call your scene parser to get your metadata and set up the scene as you see fit when new scenes are loaded in. Note that you will no longer be using .ini files and will rather just use .json files directly!

Refer to section 3.1 for more information on how to work with the parser and deal with scene changes in the codebase.

Shape tessellation

In Lab 8: Trimeshes, you should have implemented tessellation for two shapes: cube and sphere. In this project, you are expected to also include tessellation for cone and cylinder. The descriptions of these shapes remain the same as in Project 3: Intersect:

• Cube: A unit cube (i.e. sides of length 1) centered at the origin.
• Sphere: A sphere centered at the origin with radius 0.5.
• Cone: A cone centered at the origin with height 1 whose bottom cap has radius 0.5.
• Cylinder: A cylinder centered at the origin with height 1 whose top and bottom caps have radius 0.5.

The tessellation parameters should control the vertical and radial tessellations just as they do for the sphere. As a reminder, parameter 1 controls tessellation along the latitude direction while parameter 2 controls tessellation along the longitude direction.

Be especially careful when calculating normals for the tip of the cone: make sure they are always inline with one on the implicit cone, but NOT any face normal, which would result in flat shading, nor pointing straight up, which can be the edge case for your ray tracer.

Correct Cone Tip Normal Diagram

That is, the normals at the tip should be using the same equation as the normals elsewhere but its theta should step forward by . The normals elsewhere on the body should be accurate to that of the implicit cone (x and z point radially outward and y points upwards perpendicular to the slope of the cone).

We recommend working on these shapes in your lab 8 stencil prior to porting them into Project 5: Lights, Camera. This will allow you to use the visualizer to debug position and normals.

While the specifics of your tessellation code are up to you, you are expected to design your program in an extensible, object-oriented way. This means minimal code duplication and no 400 line branch structures (such as if... else... statements). You will lose points if you do not follow these guidelines.

Camera

Your camera for the raytracer only needed to produce a view matrix. For Project 5: Lights, Camera, you must produce both view and projection matrices given the scene file's camera parameters. The projection matrix is needed to convert from camera space to clip space for OpenGL to render the scene correctly.

To implement your projection matrix, you may not use glm::perspective.

Keep in mind you are able to edit the near & far plane distances in real-time. These are seen in the parameters: settings.nearPlane and settings.farPlane. Be sure to update your camera accordingly when these parameters change!

You will be expanding on your camera's functionality in the next project, so be sure to keep this in mind when implementing your camera.

Data Handling

Welcome to the meat and potatoes of this project!

You will use everything you have learned from lecture and labs to use the OpenGL pipeline to manipulate and keep track of scene data. You will take your parsed scene metadata and use it to construct all necessary VAO/VBO objects. Then you will use these in the main render loop of paintGL to finally render the scene, while integrating materials, global data, and light data as uniforms.

As far as the design goes, some questions you might want to ask are:

• How will I represent my shape data in OpenGL?
• What do my VAOs and VBOs need to be able to do/How can I generalize what I did in lab 9?
• How will I use my parsed RenderData to draw my scene in paintGL?
• How many VBO/VAOs will I need in each scene? Is it dependent on the scene?
  • Please do not include more VBOs and VAOs than necessary! For example, two separate VBOs should not store the same data. Points will be deducted for excess memory usage.

In general, filling realtime.cpp with all of your gl_____ calls is likely bad code design and will make debugging MUCH MORE DIFFICULT!

Shaders

For this project, your shader program should have the following features:

• Support for directional lights
• Ambient, diffuse, and specular intensity computation
• Final color computation integrating both object and light color
• Support for up to 8 simultaneous lights. (See subsection below)

Of course, your shaders must also perform all the necessary coordinate space conversions (think MVP matrices) as well as the illumination model computation. Think about how all of your scene data will integrate with your shaders! Which parts can you do in the Fragment shader, which parts can you do in the Vertex shader?

As such, it is important to have completed Lab 10: Shaders before attempting this part of the project.

Arrays and Structs in GLSL

In Lab 10: Shaders, you learned about various uniforms to use CPU data in a shader. When dealing with multiple identical objects, the common approach is to immediately think about arrays. In GLSL, an array of vec3s looks like this:

uniform vec3 myVectors[8];

Notice how this array is of fixed size 8, specified explicitly in code. This is because GLSL does not support dynamically sized arrays! And it's also why we require you to support an explit number of lights.

You can access element i in this array as follows:

vec3 myithElement = myVectors[i];

The next question you may have is how to actually pass data into a uniform array. For example, to fill in the element of the array with the vector (x, y, z), you would write the following:

GLint loc = glGetUniformLocation(shaderHandle, "myVectors[" + std::to_string(j) + "]");
glUniform3f(loc, x, y, z);

If you wish to get fancy, you can try using structs as well. They have to be first defined in the shader then declared as a uniform in the following manner:

struct AwesomeStruct
{
  int favoriteNumber;
  vec3 favoriteColor;
};

uniform AwesomeStruct myStruct;

Accessing the member favoriteColor from the uniform is done as such:

vec3 coolestColor = myStruct.favoriteColor;

To set the color data to a vector (r, g, b), you would write the following:

GLint loc = glGetUniformLocation(shaderHandle, "myStruct.favoriteColor");
glUniform3f(loc, r, g, b);

If you wish to read more about uniform in GLSL, check out this link!

Special tip about GLSL

Results

Here are some sample images of what your realtime renderer should be capable of by the end of this assignment.

Stencil Code

You may notice the stencil code provided is minimal--that is by design. To complete this assignment, you will need to have a good understanding of the OpenGL pipeline.

We have provided for you the following files which you will interact with:

• Realtime : A file containing the initialization of an OpenGL context as well as functions that are automatically called on certain events. These include: initializeGL, paintGL, and resizeGL.
• Settings : initialized from an .xml file along with any gui sliders/checkboxes which are updated in realtime.

And you have already written the following:

• A scene parser (lab 5)
• A basic camera class (Ray projects)
• Cube and sphere classes (lab 8)

Loading Scenes

As stated before, you will need to handle the loading of scenes using your scene parser from lab 5. We have provided for you a helper function in realtime.cpp for you to use for this purpose titled sceneChanged(). This function will be called whenever the "Upload Scene File" button is pressed and a .json file is selected. To get access to the current scenefile, you can use the settings object's sceneFilePath parameter.

Important: We will not be working with .ini files in this project! Given the real-time nature of this project, settings and parameters will be controlled by interactive UI buttons and sliders instead.

Realtime::initializeGL(), Realtime::paintGL(), & Realtime::resizeGL()

These functions are the "core" of a rendering system. They are overriden from the parent QOpenGLWidget class if you are interested.

• intializeGL() is called once near the start of the program after the constructor of Realtime has been called. It also is called before the first calls to paintGL() or resizeGL(). However, you cannot use any draw calls in this function. Rather, use it to initialize any OpenGL related information you may need, after the GLEW initialization calls as commented. Note that you cannot use any OpenGL-related functions in the constructor of this class as they are only available once GLEW has been initialized.

• paintGL() is called whenever the OpenGL context changes, i.e. when you make some state-altering OpenGL call. You won't have to worry about this for this project, but keep in mind this behavior when you add interactivity in Project 6: Action!.

• resizeGL() is called whenever the window is resized. You will need to use the input width and height to correctly update your camera.

Realtime::finish()

In OpenGL, we often use calls of the form glGen______. Just as with using the keyword new, we must delete this generated memory as well. This function, finish(), will be called just before the program exits so be sure to use it to your advantage to avoid memory leaks!

Realtime::settingsChanged()

In general, this function will be called any time a parameter of the settings is changed (via interacting with the left GUI bar) other than settings.sceneFilePath. For this project, the settings you will have to worry about are:

• settings.nearPlane: Should control your camera's near clipping plane.
• settings.farPlane: Should control your camera's far clipping plane.
• settings.shapeParameter1: Should control the tessellation parameter 1 as described in section 2.2 above.
• settings.shapeParameter2: Should control the tessellation parameter 2 as described in section 2.2 above.

Realtime::____event() Functions

These functions are all for handling interactivity which you will implement in Project 6: Action!. So do not worry about these for now!

Scenes Viewer

To assist with creating and modifying scene files, we have made a web viewer called Scenes. From this site, you are able to upload scenefiles or start from a template, modify properties, then download the scene JSON to render with your raytracer.

We hope that this is a fun and helpful tool as you implement the rest of the projects in the course which all use this scenefile format!

For more information, here is our published documentation for the JSON scenefile format and a tutorial for using Scenes.

TA Demos

Demos of the TA solution are available in this Google Drive folder titled projects_lightscamera_min.

Submission

Your repo should include a submission template file in Markdown format with the filename submission-lights-camera.md. We provide the exact scenefiles you should use to generate the outputs. You should also list some basic information about your design choices, the names of students you collaborated with, any known bugs, and the extra credit you've implemented.

For extra credit, please describe what you've done and point out the related part of your code. We have provided for you 4 different booleans in Settings for you to use for extra credit: extraCredit1, extraCredit2, extraCredit3 , and extraCredit4. These are activated by their respective GUI checkboxes. If you implement any extra features using a GUI "Extra Credit #" checkbox to be activated, please also document it accordingly so that te TAs won't miss anything when grading your assignment.

Grading

This project is out of 100 points:

• Camera data (10 points)
  • View matrix (2 points)
  • Projection matrix (8 points)
• Shape implementations (30 points)
  • Cube (5 points)
  • Sphere (5 points)
  • Cone (10 points)
  • Cylinder (10 points)
• Shaders (30 points)
  • Phong illumination (20 points)
  • Support for multiple lights (10 points)
• Tessellation (responds to changes in parameters) (10 points)
• Software engineering, efficiency, & stability (20 points)

Extra Credit

Remember that half-credit requirements count as extra credit if you are not enrolled in the half-credit course.

• Adaptive level of detail: Vary the degree of tessellation for your shapes based on:

  • The number of objects in the scene (up to 3 points): Scenes with more objects have fewer triangles, to keep the scene from getting too complex.
  • Distance from the object to the camera (up to 7 points): Objects farther from the camera should have fewer triangles than objects closer to the camera.

  Your shapes should still vary in tessellation based on the slider parameters.

• Create your own scene file (up to 2 points): Create your own scene by writing your own scene file. Refer to the provided scenefiles and to the scene file documentation for examples/information on how these files are structured. Your scene should be interesting enough to be considered as extra credit (in other words, placing two cubes on top of each other is not interesting enough, but building a snowman with a face would be interesting). If you already received extra credit for a custom scene file in the Intersect project, you cannot receive extra credit for the same scene again.

• Mesh rendering:

  • Using pre-written obj loader/parser (up to 4 points): Using a pre-existing obj loader, create trimeshes from any mesh objects and render them in their appropriate scenes.
  • Using self-written obj loader/parser (up to 8 points): To recieve a bonus 4 points, write the obj loader you use from scratch!

If you come up with your own extra credit that is not in this list, be sure to check with a TA or the Professor to verify that it is sufficient.

FAQ & Hints

Theres nothing showing up for scene ___.json, but I know it works on others:

Some scenes only have point lights in the scene; make sure the scene has directional lights.

My shapes dissappear after adding Phong lighting:

There are 2 approaches to solving this:

• Remove lighting components one at a time/build up lighting components one at a time until you see your output appear or dissapear.

• Pick out a few parameters calculated throught your phong shader and use them as the output color. Is it what you expect? These parameters could include normal vectors, diffuse dot products, light directions, or virtually anything you can think of!

My shapes look like a mess of triangles even though I know they work in the Trimeshes Lab stencil:

• Make sure you are initializing your glm::mat4s as the identity matrix explicitly. Also be sure that all of your uniforms are making it to the shader/being set properly.

Submission

Submit your GitHub repo and commit ID for this project to the "Project 5: Lights, Camera (Code)" assignment on Gradescope.

Please let us know if you find any mistakes, inconsistencies, or confusing language in this or any other CS 1230 document by filling out our anonymous feedback form.