University of Michigan Immersive 360 Spatial Audio Challenge
presented by Facebook and Real Industry
Challenge Resources
Submit your final project by 11:59 pm ET tonight.
Upload the following content to THIS FOLDER (it is imperative that every file you upload includes your team number in the file name):
1) Final encoded/muxed MKV video w/audio (Using the Facebook Encoder)
2) Screenshots of Pro Tools Session (Edit, Mix, and Clips list windows)
3) Please don’t forget to add the “_360” to your filenames:
-For 360 monoscopic videos, your MKV file name should be named filename_360.mkv
-For 360 stereoscopic videos, your MKV file name should be named filename_360_TB.mkv
If you do not name your files like this, it WILL NOT be playable.
4) Short answers (50-75 words or less) to the following questions (include screenshots where indicated, you can answer these questions in a word doc and upload the document):
- What was the treatment/vision were you going for? Did you accomplish your vision? Why/why not?
- How did the team use the Spatial Workstation tools to create the space?
*i.e What unique Spatializer parameters did you use on particular tracks, what elements did you automate
* include screenshots
-What tricks did you use to create immersion in the space? Why?
* i.e. What plugins did you use and for what unique tracks. How did the choices help your mix?
* include screenshots
- What types of audio did you use? Why?
* i.e. Ambisonics, field recordings, library content)
- What was your approach to music spatialization?
*i.e did you attach certain elements to specific actors in your scene? Did you mix in 360 or 180. Are any elements headlock
- What creative or stylistic design approaches did you take? Why?
* i.e. realistic, hyper-realistic, fiction...?
- What was the biggest challenge, technically and/or creatively?
BACKGROUND
In partnership with Facebook, you will learn to record, mix, and produce a music and spatial audio experience. Spatial audio can fully immerse a user in an experience, adding realism and directing attention within VR or 360 video. In this challenge, you will work independently in teams over 4 weeks to create 360 audio content and build a virtual reality experience.
As a result of this challenge, you will have created a vision for a 360 audio project, executed on the design and deliverables, gained a better understanding of how to design, mix and critique spatial audio experiences, and created final video and audio assets of your project that are ready for distribution across multiple platforms.
Instructions: Please read each step carefully.
Step 1 Form a team.
Team Captain
Each team must have a captain with experience using Pro Tools. The captain has been given a copy of Pro Tools | Ultimate and will lead the team in spatializing audio in the 360 session. (As a recap, here are the Setup Instructions for Team Captains to download, install, and setup Pro Tools and the Spatial Audio Workstation Tools.)
It is mandatory that each team have a captain as described above, but the following roles will also be available on each team:
Sound Designer
Responsible for recording original assets and/or choosing assets from sound libraries. Record on your phone or on your favorite handheld recorder. Create, craft, and integrate sounds using any DAW. The sound designer will work with the Team Captain (Pro Tools engineer) to integrate and mix sounds.
Music Supervisor
Responsible for creating, curating, or composing music. Supervisors choose music from a library of immersive music stems and mixes, compose, arranging music for 360 audio, and composing, remixing, or editing for the desired music. The music supervisor will integrate final music selections with the Pro Tools engineer.
Step 2 Every team member must register at the bottom of this page.
Step 3 Start your planning!
Make a copy of this Google Doc and start editing your Sound Production Template.
This document helps your team organization your production process.
Step 4 Choose and download a 4K video. This will also be the video you use for side-loading and final submission. (See also: “Prepare to Export and Submit.”)
Your options are:
Airfield- A landing strip at an airport
Cable Car- City street with cable cars and pedestrians passing through
City Pier- Pier in an urban area with birds flying, boats swaying and cars driving by in the distance
City Night- City street with moderate traffic and pedestrians
Overpass- Pedestrian overpass with light traffic below
Pacific Pier- Ocean pier with waves crashing to shore and a few onlookers and a bird
Shorebird- Walking path along the ocean at sunset, birds flying and pedestrians passing by
*None of the videos have sound
Step 5 Download this Pro Tools demo session template. This is the demo session that Jon demoed at our session.
AFTER THE KICKOFF EVENT
Step 6 Download, login, and setup your equipment.
SIgnup for the facebook group
All support will be supported in our Immersive 360 Challenge Facebook group. Send a request to join the group and you’ll be see questions asked by your peers and get quick responses to your own questions.
Pro Sound Effects
All team members have access to the Pro Sound Effects Library.
Pro Sound Effects created a login for each team to access the sounds effects library. You can use this to pull different sounds (including ambisonic sounds) for your project.
Login page for Pro Sound Effects
Login: umich360-NN@prosoundeffects.com
Temporary Password: umich360
**The bold numbers are your team number. If you are in team 1, your login is as displayed above ‘01’.University of Michigan Overview of how to use Pro Sound Effects
Contact Julia@realindustry.org if you need support on logging in.
DOWNLOAD YOUR ASSETS
Pro Tools session templates are provided for the videos your team selected.
Videos included with the Pro Tools sessions are DNxHR files; they are only compatible with Pro Tools. The are not compatible nor playable on the Oculus Go. When you side-load onto your Oculus Go, you need include the 4K MP4 files linked in the PREPARE TO EXPORT AND SUBMIT section below.Music stems for you to mix, edit, and experiment. (Or create your own music!)
In this folder you will find genre-specific subfolders with stereo preview files as well as stems for each preview.
SETUP YOUR OCULUS GO
Oculus Go: Immersive 360 Challenge Setup Instructions
Each team was loaned an Oculus Go for the duration of the challenge. Keep the box and all materials. You will use the headset to test out and experience what you create
CREATE YOUR CONTENT
Step 7 OK, time to get to work. Over the next 3+ weeks, you will add sound, music, and content to create your immersive experience.
Based on the film and music production best practices, we recommend that all teams follow this agile, iterative process:
Production Stages:
Pre-Production:
Define the Artistic Vision
Scheduling and Logistics
Asset Cataloging (what work needs to be done?)
Creation:
Recording (Foley, studio, field recording, etc)
Curation (choosing assets from content libraries)
Synthesis (creation and blending of assets)
Editing (evaluate, trim, transform)
Implementation:
Integration into Pro Tools session
Arrangement
Mixing
Optimization
Preview on Oculus Go
Rinse and Repeat Implementation until you run out of time.
Return to Creation as needed.
Delivery
Final assets delivered
Documentation (metadata, paperwork)
PREPARE TO EXPORT AND SUBMIT
Step 8 Prepare for side-loading to your Oculus Go.
When encoding your video, make sure to use the 4K MP4 video of your project here.
You can find instructions for setting up your Oculus Go here and instructions for uploading your content here.
Criteria
Mix Quality
- Is the mix clear and dynamic? If not, explain why (i.e. over compressed, distorted, muddy...).
- Does the mix playback well on device and in headphones (i.e. is it too loud or too quiet, thin or bass heavy)?
- Does anything break story (i.e. sync issues, volume inconsistencies, over all loudness control)?
- What other mix choices did or did not work?
Spatialization
- Were there particular localization choices that guided you in a specific direction? How effective was it?
- How accurate were spatialization choices (i.e. distance, azimuth, spread. could you tell where objects with sound where coming from)?
- Did the piece's spatialization feel immersive (how deep did ambiences feel, was there a sense of space)?
- What other spatialization choices did or did not work.
Creative Design
- Where you convinced by the design choices? Why/Why not?
- What interesting areas of design stood out?
- What choices helped to tie the story together?
- Did the sound design choices make sense? Why/Why not?
- Did the finished project feel polished (i.e. where there gaps in design, or too much, just right)?
- What other aspects of the piece did, or did not work?
Key dates
In the last week of the challenge there will be a Dropbox link here to submit your project.
Kick-off event: Thursday, January 24th, 6:00 PM - 9:00 PM ET
Submissions due: Thursday, February 21st, 11:59 PM ET
Peer viewing session: Friday, February 22, 2019, 1:00 PM - 4:00 PM ET
Facebook feedback session: Friday, February 22, 2019, 1:00 PM - 4:00 PM ET
Tools
Pro Sound Effects Library
Pro Sound Effects has provided a cloud login for each team to access the sounds effects library. You can use this to pull different sounds (including ambisonic sounds) for your project. You can access the library here (https://library.prosoundeffects.com). Your login is the following*:
Login: umich360-nn@prosoundeffects.com*
Temporary Password: umich360
**The bold numbers are your team number. If you are in team 1, your login is as displayed above ‘01’.
Avid Pro Tools
The captain on each team has received a generously donated NFR $999 copy of Avid Pro Tools l Ultimate. If you are a team captain and have not received your access code, please reach out to julia@realindustry.org.
Facebook Challenge Support
Throughout the challenge you can get support in this Facebook group. Send a request to join the group and you’ll be able to see questions asked by your peers and get quick responses to your own questions.
resources
Prizes & MENTORSHIP PROCESS
ALL TEAMS that submit a project will receive a one-on-one 30-minute feedback session with experts in sound design from the Facebook Sound Design team!
1 “People’s Choice” team will be select by a student Peer Review process!
Each team member (up to 4 max) will receive an Oculus Go.