bbc
gadgets to use
geeky gadgets
gizmodo
jalopnik
jezebel
kotaku
lifehacker
ny times
pocket lint
readwrite
tech republic
tech world
techcrunch
technology
the inventory
the next web
venture beat
wired
count
sauces
help
I'm confused... What am I doing here?

Choose which sources you wish to remain and you're all set. Use the buttons to turn sources on and off.

What do the different colours mean?

Depending on if any articles/links are visible on the page there are 4 modes to show the state of your chosen sources.

on / visible on page

off / visible on page

on / not visible on page

off / not visible on page

New York City bans Zoom in schools citing security concerns - techcrunch

New York City bans Zoom in schools citing security concerns

As schools lie empty, students still have to learn. But officials in New York City say schools are not permitted to use Zoom for remote teaching, citing security concerns with the video conferencing service. “Providing a safe and secure remote learning experience for our students is essential, and upon further review of security concerns, schools […]
techcrunch - 1 hour ago
R&D Roundup: Ultrasound/AI medical imaging, assistive exoskeletons and neural weather modeling - techcrunch

R&D Roundup: Ultrasound/AI medical imaging, assistive exoskeletons and neural weather modeling

In the time of COVID-19, much of what transpires from the science world to the general public relates to the virus, and understandably so. But other domains, even within medical research, are still active — and as usual, there are tons of interesting (and heartening) stories out there that shouldn’t be lost in the furious […]
techcrunch - 4 hours ago
Original Content podcast: ‘The Platform’ offers a gruesome metaphor for capitalism - techcrunch

Original Content podcast: ‘The Platform’ offers a gruesome metaphor for capitalism

“The Platform” is not a subtle movie. That’s true of its approach to horror, with intense, bloody scenes that prompted plenty of screaming and pausing from your hosts at the Original Content podcast. It’s also true of its thematic material — right around the time one of the characters accuses another of being communist, you’ll […]
techcrunch - 23 hours ago
This Week in Apps: Zoom has issues, Pinterest founder’s new COVID-19 research app, record Q1 spending - techcrunch

This Week in Apps: Zoom has issues, Pinterest founder’s new COVID-19 research app, record Q1 spending

This week... how the COVID-19 outbreak is impacting apps and the wider mobile app industry, more leaks about the new version of iOS, Apple bought Dark Sky, and more.
techcrunch - 1 day ago
Coronavirus Lockdown: List of Smartphone Companies Offering Warranty Extension in India - gadgets to use

Coronavirus Lockdown: List of Smartphone Companies Offering Warranty Extension in India

Prime Minister Narendra Modi announced a nationwide lockdown for 21 days in a bid to try to stop the spread of coronavirus pandemic in India. Due to this ongoing lockdown, many smartphone companies in India have announced a warranty extension. Smartphone maker companies including Samsung, OnePlus, Realme and others have extended warranties on their products […] The post Coronavirus Lockdown: List of Smartphone Companies Offering Warranty Extension in India appeared first on Gadgets To Use.
gadgets to use - 1 day ago
3 Best Magic Video Effects Apps for Android - gadgets to use

3 Best Magic Video Effects Apps for Android

Magic video creators are a great way to unleash your creativity and kill boredom, especially in these quarantine days. But unfortunately, the majority of apps available in the market often turn out to be rogue and are quite the opposite of what’s advertised. Therefore, to save your time, we are here with the three best […] The post 3 Best Magic Video Effects Apps for Android appeared first on Gadgets To Use.
gadgets to use - 1 day ago
Zoom admits some calls were routed through China by mistake - techcrunch

Zoom admits some calls were routed through China by mistake

Hours after security researchers at Citizen Lab reported that some Zoom calls were routed through China, the video conferencing platform has offered an apology and a partial explanation. To recap, Zoom has faced a barrage of headlines this week over its security policies and privacy practices, as hundreds of millions forced to work from home […]
techcrunch - 1 day ago
Before suing NSO Group, Facebook allegedly sought their software to better spy on users - techcrunch

Before suing NSO Group, Facebook allegedly sought their software to better spy on users

Facebook's WhatsApp is in the midst of a lawsuit against Israeli mobile surveillance outfit NSO Group. But before complaining about the company's methods, Facebook seems to have wanted to use them for its own purposes, according to testimony from NSO founder Shalev Hulio.
techcrunch - 1 day ago
Zelos is like a cross-game battle pass, rewarding you for completing challenges in games you already play - techcrunch

Zelos is like a cross-game battle pass, rewarding you for completing challenges in games you already play

People seem to love the concept of the battle pass. Largely popularized by Fortnite, battle passes reward players for playing well, and playing often. The better you do, the more XP you earn; the more XP you earn, the more stuff (new looks for your character, or victory dances to fire off at the end […]
techcrunch - 1 day ago
Google research makes for an effortless robotic dog trot - techcrunch

Google research makes for an effortless robotic dog trot

As capable as robots are, the original animals after which they tend to be designed are always much, much better. That's partly because it's difficult to learn how to walk like a dog directly from a dog — but this research from Google's AI labs make it considerably easier.
techcrunch - 1 day ago
Want to survive the downturn? Better build a platform - techcrunch

Want to survive the downturn? Better build a platform

When you look at the most successful companies in the world, they are almost never just one simple service. Instead, they offer a platform with a range of services and an ability to connect to it to allow external partners and developers to extend the base functionality that the company provides. Aspiring to be a […]
techcrunch - 1 day ago
Pandemic puts the brakes on micromobility - techcrunch

Pandemic puts the brakes on micromobility

As of this writing, nearly a million people globally have been infected with the novel coronavirus and 50,322 have died. Healthcare systems are overwhelmed, consumers and profiteers are hoarding supplies and some service workers have launched strikes while many others have been let go. In the world of micromobility, we’ve seen Bird lay off hundreds […]
techcrunch - 1 day ago
Zoom will enable waiting rooms by default to stop Zoombombing - techcrunch

Zoom will enable waiting rooms by default to stop Zoombombing

Zoom is making some drastic changes to prevent rampant abuse as trolls attack publicly-shared video calls. Starting April 5th, it will require passwords to enter calls via Meeting ID, since these may be guessed or reused. Meanwhile, it will change virtual waiting rooms to be on by default so hosts have to manually admit attendees. […]
techcrunch - 1 day ago
Stocks drop as unemployment spikes - techcrunch

Stocks drop as unemployment spikes

Stocks fell in regular trading Friday, as all major American indices fell in the wake of a broadly negative jobs report. With more than 700,000 jobs lost in the March data, unemployment in the United States rose from 3.5% to 4.4%. The markets have been bracing for widespread job losses due to the continued fallout […]
techcrunch - 1 day ago
The pandemic is already reshaping tech’s misinformation crisis - techcrunch

The pandemic is already reshaping tech’s misinformation crisis

Since 2016, social media companies have faced an endless barrage of bad press and public criticism for failing to anticipate how their platforms could be used for dark purposes at the scale of populations—undermining democracies around the world, say, or sowing social division and even fueling genocide. As COVID-19 plunges the world into chaos and […]
techcrunch - 1 day ago
‘Artemis Fowl’ is skipping theaters for Disney+ - techcrunch

‘Artemis Fowl’ is skipping theaters for Disney+

With movie theaters largely closed due to the COVID-19 pandemic, Disney is pushing back its slate of upcoming films. And at least one movie won’t be making it into theaters at all, with “Artemis Fowl” heading straight to streaming instead. The company announced today that that the film will debut exclusively on Disney+, and that […]
techcrunch - 1 day ago
Google rolls back SameSite cookie changes to keep essential online services from breaking - techcrunch

Google rolls back SameSite cookie changes to keep essential online services from breaking

Google today announced that it will temporarily roll back the changes it recently made to how its Chrome browser handles cookies in order to ensure that sites that perform essential services like banking, online grocery, government services and healthcare won’t become inaccessible to Chrome users during the current COVID-19 pandemic. The new SameSite rules, which […]
techcrunch - 1 day ago
How Homage is tackling Southeast Asia’s growing eldercare need - techcrunch

How Homage is tackling Southeast Asia’s growing eldercare need

The world’s population is aging, but the needs of elderly people are still being underserved. A United Nations report found that older people make up more than one-fifth of the population in 17 countries, and by 2100, a majority of the world’s population, or 61%, will be aged 60 and above. One of the most […]
techcrunch - 2 days ago
GM and Honda are co-developing two new electric vehicles due to arrive in 2024 - techcrunch

GM and Honda are co-developing two new electric vehicles due to arrive in 2024

GM and Honda will jointly develop two new electric vehicles slated for 2024, the latest move by the two automakers to deepen their existing partnership. Under the plan, the automakers will focus on their respective areas of expertise. Honda will design the exterior and interiors of the new electric vehicles. GM will contribute its new […]
techcrunch - 2 days ago
Insight closes $9.5B fund to help support portfolio companies through the pandemic crunch - techcrunch

Insight closes $9.5B fund to help support portfolio companies through the pandemic crunch

We’re now several weeks into what is has become a very big dip for the global economy due to the coronavirus pandemic, but amidst that, we are seeing are some notable pockets of investment activity emerging that will help shape how the future startup landscape will look. Today one of the biggest venture capital firms […]
techcrunch - 2 days ago
LOAD MORE TECH
bbc
gadgets to use
geeky gadgets
gizmodo
jalopnik
jezebel
kotaku
lifehacker
ny times
pocket lint
readwrite
tech republic
tech world
techcrunch
technology
the inventory
the next web
venture beat
wired
count
sauces
help
I'm confused... What am I doing here?

Choose which sources you wish to remain and you're all set. Use the buttons to turn sources on and off.

What do the different colours mean?

Depending on if any articles/links are visible on the page there are 4 modes to show the state of your chosen sources.

on / visible on page

off / visible on page

on / not visible on page

off / not visible on page

techcrunch
How Spotify ran the largest Google Dataflow job ever for Wrapped 2019

How Spotify ran the largest Google Dataflow job ever for Wrapped 2019

techcrunch - 1 month ago

In early December, Spotify launched its annual personalized Wrapped playlist with its users’ most-streamed sounds of 2019. That has become a bit of a tradition and isn’t necessarily anything new, but for 2019, it also gave users a look back at how they used Spotify over the last decade. Because this was quite a large job, Spotify gave us a bit of a look under the covers of how it generated these lists for its ever-growing number of free and paid subscribers.

It’s no secret that Spotify is a big Google Cloud Platform user. Back in 2016, the music streaming service publicly said that it was going to move to Google Cloud, after all, and in 2018, it disclosed that it would spend at least $450 million on its Google Cloud infrastructure in the following three years.

It was also back in 2018, for that year’s Wrapped, that Spotify ran the largest Google Cloud Dataflow job ever run on the platform, a service the company started experimenting with a few years earlier. “Back in 2015, we built and open-sourced a big data processing Scala API for Apache Beam and Google Cloud Dataflow called Scio,” Spotify’s VP of Engineering Tyson Singer told me. “We chose Dataflow over Dataproc because it scales with less operational overhead and Dataflow fit with our expected needs for streaming processing. Now we have a great open-source toolset designed and optimized for Dataflow, which in addition to being used by most internal teams, is also used outside of Spotify.”

For Wrapped 2019, which includes the annual and decadal lists, Spotify ran a job that was five times larger than in 2018 — but it did so at three-quarters of the cost. Singer attributes this to his team’s familiarity with the platform. “With this type of global scale, complexity is a natural consequence. By working closely with Google Clouds engineering teams and specialists and drawing learnings from previous years, we were able to run one of the most sophisticated Dataflow jobs ever written.”

Still, even with this expertise, the team couldn’t just iterate on the full data set as it figured out how to best analyze the data and use it to tell the most interesting stories to its users. “Our jobs to process this would be large and complex; we needed to decouple the complexity and processing in order to not overwhelm Google Cloud Dataflow,” Singer said. “This meant that we had to get more creative when it came to going from idea, to data analysis, to producing unique stories per user, and we would have to scale this in time and at or below cost. If we werent careful, we risked being wasteful with resources and slowing down downstream teams.”

To handle this workload, Spotify not only split its internal teams into three groups (data processing, client-facing and design, and backend systems), but also split the data processing jobs into smaller pieces. That marked a very different approach for the team. “Last year Spotify had one huge job that used a specific feature within Dataflow called “Shuffle.” The idea here was that having a lot of data, we needed to sort through it, in order to understand who did what. While this is quite powerful, it can be costly if you have large amounts of data.”

This year, the company’s engineers minimized the use of Shuffle by using Google Cloud’s Bigtable as an intermediate storage layer. “Bigtable was used as a remediation tool between Dataflow jobs in order for them to process and store more data in a parallel way, rather than the need to always regroup the data,” said Singer. “By breaking down our Dataflow jobs into smaller components — and reusing core functionality — we were able to speed up our jobs and make them more resilient.”

Singer attributes at least a part of the cost savings to this technique of using Bigtable, but he also noted that the team decomposed the problem into data collection, aggregation and data transformation jobs, which it then split into multiple separate jobs. “This way, we were not only able to process more data in parallel, but be more selective about which jobs to rerun, keeping our costs down.”

Many of the techniques the engineers on Singer’s teams developed are currently in use across Spotify. “The great thing about how Wrapped works is that we are able to build out more tools to understand a user, while building a great product for them,” he said. “Our specialized techniques and expertise of Scio, Dataflow and big data processing, in general, is widely used to power Spotifys portfolio of products.”

sauce: techcrunch
CLOSE