Peex let me remix augmented reality sounds at Elton John’s concert

Click here to view original web page at venturebeat.com

Peex is a United Kingdom tech startup that has invented a concert audio remixing system that lets fans control and customize the audio at live concerts, allowing them to hear the music with close to studio-quality remixed sounds.

Peex Live promises to deliver perfect sounds, no matter where your seats are in a big stadium. John’s final tour before retirement was a great chance to test the Peex technology on the road in America.

I tried out Peex Live at last week’s concert at the Chase Center in San Francisco, which featured Elton John’s Farewell Yellow Brick Road Tour. And it really worked. I felt like I could hear more of the legendary singer’s voice and his piano skills, instead of being overwhelmed by the drums, bass guitar, keyboards, and guitar. To me, it was an amazing experience that made me feel like John still had a solid singing voice, as I remembered from his records decades ago.

Normally, the concert sounds I hear — like I remember from John’s previous tour in San Jose, California — are muddy. They blur together, and his voice gets drowned out by other noises. But to me, the remix that I created with the Peex headset and mobile app, sounded great. On the other hand, my wife couldn’t tolerate the loud sounds in the remix, and she had to take the headset off.

Therein lies the complexity of the concert music problem that Peex is trying to fix. Every listener is different. Graham Tull, chief technology officer and cofounder at Peex, calls it “augmented audio reality.” During John’s goodbye tour, Peex is renting out audio wearables and headsets to concert-goers as part of its plan to take the technology wide. The rental fee for Peex is $15 per show, and that is generating some revenue for the company.

“Every seat in the house is the best seat in the house,” Tull said.

In an interview at the concert, Tull told me that John’s concert tour, which started September 4 in Salt Lake City, is a big chance for the company to show off the working wearable technology, which has been tuned over the years. His team has been working with John for the past four years to try to revolutionize the way fans listen to live music.

When you look at the Peex Live app, it’s simple enough. You download it on iOS or Android and then pair it via Bluetooth with the Peex rX wearable, which is round medallion that hangs around your neck on a string. You plug in earphones into it and control the sounds with the app.

Peex Live’s patented technology lets fans their own enhanced five-channel mix of the live concert sound. But you can still hear the ambient noise of the crowd and the concert through the headphones. That preserves the energy of the performance, Tull said.

How it got started

Tull told me that the idea surfaced in 2013 and the founders created their startup in 2014. The team has done about 100 shows to date in countries around the world. Tull used to play in a band and his close friend worked in John’s production team. The self-financed team grew to about 25 people.

That contact enabled the team to get an inside track with John and hook up with his band for the early development of the technology. The team had to combine sound engineers, record producers, and musicians.

How it works

The farewell tour is the first chance for U.S. concert goers to hear the Peex technology. Peex takes the audio directly from the stage — both analog and digital tracks — and converts it into a digital audio source. That sound goes to the speakers and to the musicians’ headsets.

But Peex also gets a split of the sound tracks. The submixes are brought into the Peex system. The five channels are not truly solo, as they all have some base level of sound, with one instrument or vocal as the primary sound in the channel. With one prominent sound, you can emphasize that element of the mix.

The audio from the stage is converted into radio packets, which are broadcast through transmitters on the lighting rig. Then a proprietary version of the 5 gigahertz Wi-Fi system in the stadium transmits it in an encrypted stream of digital data to the wearable, which has a chip that decodes it and sends it into the earphones.

But the system also makes a calculation about when to send it in to your ears, based on where you are and when you are expected to hear the natural sound coming from the concert’s speakers. A microphone in the wearable helps identify the sound coming from the speakers. With this added calculation, you don’t hear multiple sets of sounds, Tull said.

“It brings the two sounds into perfect synchronization and is very accurate, to roughly 1.3 meters of distance,” he said. “If you walk to the back for a beer, it’s going to move with you. That gets rid of the time delay so that the audio overlays what is happening in the room and you hear high-definition audio.”

Tull draws an analogy to the huge video screens, which let you see John on stage up close, even if your seat is far away. Likewise, Peex lets you experience the same high-definition audio no matter where you are in the soundscape of the stadium.

You could get better sound quality through noise cancellation headsets, but Tull said the team decided against that because it isolates you from the concert.

“It’s a bit like wearing headphones on an airplane,” he said. “You can’t hear the crowd. That’s why the delay estimation and synchronizing sounds sources is vital. The different sounds become indistinguishable from each other.”

If your phone battery dies

One of the smart things is that the phone merely serves as the control mechanism. The system does not use your phone to send music to your ears. That would introduce too much latency and drain your battery.

Rather, you use the phone to control the different channels. If your phone battery dies, then the system will stay with whatever the last setting was and you will still hear music through the headphones.

“We designed the battery for sessions that could last seven or eight hours, in case you were at something like a music festival,” Tull said.

That said, my phone battery was pretty close to dead from using it all day, so I had to conserve my battery and refrain from taking too many videos.

My own remix experience

For me, hearing was believing.

On the companion app, there are five bars representing different channels. Each channel has an image on it, like Elton John for vocals or Davey Johnstone for guitar. You use your finger to slide the image on the bar upward to increase the sound of that channel, and you can lower the others to highlight it further. I kept John’s vocals high and had everything else lower. I kept the drums lowest of all. And the vocals sounded crystal clear to me.

That was something new. John signs so fast that I often can’t recognize the words he is singing, even if I remember the song. With the app, it was quite easy for me to become a sound engineer. If I turned up the guitar, the guitar chords would slowly become sharper. If I pulled the earphones out of my ears, It felt like the noise amplified and the distinct musical sounds began to vanish.

As I heard my favorite songs — Daniel, Rocket Man, Crocodile Rock, Candle in the Wind, Your Song, and Goodbye Yellow Brick Road — I was transported back to my youth. I even recorded the songs on my iPhone, as you can see with the embedded video. I tried it out close to the front and in the back.

During one of Johnstone’s guitar solos, I turned up the guitar channel to clearly hear his twanging. Then I could tap the memory button and go back to my original sound settings.

I think you might agree that the sound of the concert songs isn’t really spectacular. But to my ears, with Peex, it was good. There was no latency, or delay, as the sounds moved through the Peex system and into my ears.

My wife’s harder time

My wife, on the other hand, was not able to lower the volume to an acceptable level, and so she could not tolerate it. If she raised the vocals high and lowered the other tracks lower, she found that the volume of the higher vocals seemed to simply just add on top of the other tracks. the other tracks, in other words, did not go low enough in volume for her. It was too much volume. Even if she lowered the volume with the minus sign. So the base level of sound was high and she could not lower it enough to make the rest of the broadcast tolerable.

In response, Tull said, ” Obviously, everyone has different hearing sensitivity and we have calibrated the Peex system to operate within lower and upper limits designed to give sufficient volume range without making the volume uncomfortable. There is a lower volume limit set in the system, which requires the user to manually override to reach the higher range. Having this limit set on would have helped your wife experience a more acceptable volume level for her particular hearing sensitivity.”

He added, “To the point about overcoming other instrumentation (channels) when boosting e.g. the vocal channel. The volume scale of the mixer does not go to zero as the ambient room sound is always the loudest element. This can, at times, reach 100dB(SPL) at rock concerts. So even with four faders on the lowest setting, there is still perceptible sound from these respective channels. That said, most people would struggle to perceive this level of volume above the natural room sound, hence my early point about hearing sensitivity is different for different people. We also mix a proportion of the overall mix into each channel so that there is never a fully isolated element. This is to ensure that a natural mix occurs in sympathy with the ambient room mix, rather than creating something that sits distinctly isolated from the overall sound. So to be clearer, our intention is to allow users to have a facility to enhance and augment their personal sound experience rather than to radically change the intended delivery of the songs performed on stage. I hope that makes some sense.”

Sound dropouts

There were a couple of times when the sound went mute on me as if the wireless tech lost its connection. I wondered if more transmitters throughout the stadium would help overcome this problem.

Tull said, “The Peex system uses license-free frequencies which are also used by Wi-Fi systems. This means we very occasionally get co-channel interference, which can cause a momentary drop out in audio. This is usually a very rare event over the course of the show. We set the system up to deliver a real-time audio stream to any part of the arena, but inevitably there can be areas where the signal is less strong.”

He added, “You rightly assume that more transmitters would overcome this issue, but of course, we have to strike a balance between an excellent level of performance, the time taken to load in and load out a show and the required channels needing to be made available for the Peex transmissions. Unlike other wireless data services (e.g. browsing a website via Wi-Fi) which can buffer and modulate their data rates according to available bandwidth, we must send continuous real-time data, which is a very demanding requirement in a hostile RF environment.”

He added, “While video can miss a frame or so and you won’t notice, the hard thing about audio in real-time is that the human brain is less forgiving when it comes to filling in the gaps.”

More concerts to come

There are still plenty of dates left in the tour, from an October 19 show in Milwaukee Wisconsin to the final show on November 16 in Nassau Coliseum in Long Island, New York. It picks up again in 2021. Peex’s founding team has close connections to John, but John himself hasn’t invested in the company.

Tull said the comments coming in from fans is “fabulous.”

“We are getting the public to understand it is available, and then we’ll gradually widen the usage,” he said. “More bands, more venues will come.”

Over time, Peex might offer subscriptions for devices so people can take their own devices to multiple shows.

“We’re not charging anything for the artists,” Tull said. “This is all about how the fan can have the perfect experience.”

Conclusion

Peex is going to be a very big thing when it reaches its full maturity. I would gladly wear it and pay an extra price, particularly if means hearing much better sound. It may depend on venues having good Wi-Fi technology, and that is happening. But overcrowded wireless networks may cause the occasional dropouts. As 5G technology comes, this technology should get a big boost. And as my wife can attest, it won’t work for everybody. My hearing isn’t that great. But Peex overcame my skepticism. I might be open to going to more concerts.

Disclosure: The Peex folks gave me tickets to the Elton John concert. Our coverage remains objective.

Click here to view full article