A load balancer that learns, WebTorch

In my previous blog post “How I stopped worrying and embraced docker microservices” I talked about why Microservices are the bees knees for scaling Machine Learning in production. A fair amount of time has passed (almost a year ago, whoa) and it proved that building Deep Learning pipelines in production is a more complex, multi-aspect problem. Yes, microservices are an amazing tool, both for software reuse, distributed systems design, quick failure and recovery, yada yada. But what seems very obvious now, is that Machine Learning services are very stateful, and statefulness is a problem for horizontal scaling.

Context switching latency

An easy way to deal with this issue is understand that ML models are large, and thus should not be context switched. If a model is started on instance A, you should try to keep it on instance A as long as possible. Nginx Plus comes with support for sticky sessions, which means that requests can always be load balanced on the same upstream a super useful feature. That was 30% of the message of my Nginxconf 2017 talk.

The other 70% of my message was urging people to move AWAY from microservices for Machine Learning. In an extreme example, we announced WebTorch, a full-on Deep Learning stack on top of an HTTP server, running as a single program. For your reference, a Deep Learning stack looks like this.

Pipeline required for Deep Learning in production.
What is this data, why is it so dirty, alright now it’s clean but my Neural net still doesn’t get it, finally it gets it!

Now consider the two extremes in implementing this pipeline;

  1. Every stage is a microservice.
  2. The whole thing is one service.

Both seem equally terrible for different reasons and here I will explain why designing an ML pipeline is a zero-sum problem.

Communication latency

If every stage of the pipeline is a microservice this introduces a huge communication overhead between microservices. This is because very large dataframes which need to be passed between services also need to be

  1. Serialized
  2. Compressed (+ Encrypted)
  3. Queued
  4. Transfered
  5. Dequeued
  6. Decompressed (+ Decrypted)
  7. Deserialized

What a pain, what a terrible thing to spend cycles on. All of these actions need to be repeated every time the microservice limit is crossed. The horror, the terrible end-to-end performance horror!

In the opposite case, you’re writing a monolith which is hard to maintain, probably you’re either using uncomfortable semantics either for writing the HTTP server or the ML part, can’t monitor the in between stages etc. Like I said, writing a ML pipeline for production is a zero-sum problem.

An extreme example; All-in-one deep learning

Venn diagram of torch, nginx
Torch and Nginx have one thing in common, the amazing LuaJIT
That’s right, you’ll need to look at your use case and decide where you draw the line. Where does the HTTP server stop and where does the ML back-end start. If only there was a tool that made this decision easy and allowed you to even go to the extreme case of writing a monolith, without sacrificing either HTTP performance (and pretty HTTP server semantics) or ML performance and relevance in the rapid growing Deep Learning market. Now such a tool is here (in alpha) and it’s called WebTorch.

WebTorch is the freak child of the fastest, most stable HTTP server, nginx and the fastest, most relevant Deep Learning framework Torch.

Now of course that doesn’t mean WebTorch is either the best performance HTTP server and/or the best performing Deep Learning framework, but it’s at least worth a look right? So I run some benchmarks, loaded the XOR neural network found at the torch training page. I used another popular Lua tool, wrk to benchmark my server. I’m sending serialized Torch 2D DoubleTensor tensors to my server using POST requests to train. Here’s the results:

Huzha! Over 1000 req/sec on my Macbook air, with no Cuda support and 2 Intel cores!

So there, plug that into a CUDA machine and see how much performance you squeeze out of that bad baby. I hope I have convinced you that sometimes, mixing two great things CAN lead to something great and that WebTorch is an ambitious and interesting open source project! Check out the Github repo and give it a star if you like the idea.

https://github.com/UnifyID/WebTorch

And hopefully, in due time it will become a fast, production level server which makes it easy for Data Scientists to deploy their models in the cloud (do people still say cloud?) and devOps people to deploy and scale.

Possible applications of such a tool include, but not limited to:

  • Classification of streaming data
  • Adaptive load balancing
  • DDoS attack/intrusion detection
  • Detect and adapt to upstream failures
  • Train and serve NNs
  • Use cuDNN, cuNN and cuTorch inside NGINX
  • Write GPGPU code on NGINX
  • Machine learning NGINX plugins
  • Easily serve GPGPU code
  • Rapid prototyping Deep Learning solutions

Maybe your own?

Our Pledge to Inclusion and Diversity: 1 Year Later

Lack of diversity in tech has been a long-standing problem, but in recent months it’s become increasingly apparent that inclusion is more than an aspirational need. Diversity is the DNA that creates robust, flourishing environments primed for tough conversations and progressive thinking at UnifyID.

Last June, UnifyID was one of 33 companies that signed the White House Tech Inclusion Pledge on the eve of President Obama’s Global Entrepreneurship Innovation Summit 2016 to ensure that our employees reflect the diverse nature of the American workforce.

Although UnifyID is a small startup, we still want to lead in all areas of our business—and diversity is no exception. As an inaugural signatory of this agreement, the first of its kind, we proudly reaffirm our commitment to being an industry leader in promoting inclusion for all.

Our team on a normal day in the office.

The pledge was three-part, with the central aim of increasing representation of underrepresented groups:

“Implement and publish company-specific goals to recruit, retain, and advance diverse technology talent, and operationalize concrete measures to create and sustain an inclusive culture.”

This was a task we have invested significant time and effort into accomplishing, particularly in our recruitment operations. Many job seekers and experts alike have criticized the inconsistent process around the technical interview, noting its irrelevance to the workplace and its unnecessary biases against women. Taking into account these guidelines from Code2040, a collaborating organization of the Tech Inclusion Pledge, we’ve created a low stress, context-relevant, and fun language-agnostic technical challenge to improve the non-biased screening in our recruiting process.

“Annually publish data and progress metrics on the diversity of our technology workforce across functional areas and seniority levels.”

It is important to us that we are transparent about our gender, racial, and ethnic data because diversity and inclusion is a core part of our company mission to be authentic, be yourself. As such, this report is our first attempt at this, and we hope to make future updates more frequently.

On our team, 70 percent are people of color and 24 percent are women. Immigrants make up a significant part of the American workforce, and we are also proud to call UnifyID the workplace of immigrants who collectively represent 17 nationalities (including our interns). Paulo, one of our machine learning engineers, has quipped, “the office sometimes feels like a Model UN conference!” While our size makes us unable to release more detailed breakouts (we respect employee privacy), we will continue to release diversity data in a timely and transparent fashion.

“Invest in partnerships to build a diverse pipeline of technology talent to increase our ability to recognize, develop and support talent from all backgrounds.”

Here in the Bay Area, we are surrounded by terrific organizations that support underrepresented groups in tech, and we’ve been fortunate to be involved in these events. Some of these events include the Out for Undergrad (O4U) annual Tech Conference, which allowed us to connect with many high-achieving LGBTQ+ undergraduates from across the country, as well as the Y Combinator-hosted Female Founders Conference, or even SF Pride last month!

Our head of Product, Grace Chang, at last year’s Out for Undergrad (O4U) Tech Conference!

Diversity strengthens us as a company and as a country, so this remains one of our foremost priorities as we continue to grow (we’re hiring) and we hope to see improvement in our workplace and in the industry as a whole. We are thrilled that today, the number of companies that have signed the pledge has risen to 80.

We encourage more companies to sign this Tech Inclusion Pledge here.

Housewarming Party at UnifyID!

At a startup like UnifyID, it’s amazing how much can change over a few weeks’ time. What’s even more incredible, though, is how unpredictable the catalyst for that change can be. It’s been almost four months since we were unanimously crowned winners of RSA’s Innovation Sandbox, and the positive response we’ve received since has been overwhelming.

Last Friday, we hosted a housewarming party at our new SoMa office celebrating all the good work we’ve done including: wrapping up the Spring AI Fellowship, winning other competitions, kicking off new partnerships, welcoming a new batch of summer interns (pictured below), and a special announcement from founders John Whaley and Kurt Somerville!

We’re so grateful to everyone who attended the Housewarming and all who continue to support the mission of our work.

Interested in learning more about this secret announcement? Join the team, lead the frontier in how people interact seamlessly with technology, and let’s change authentication forever.

Photo credits: Karina Furhman

UnifyID Takes Home the Win at SXSW!

Our first trip to SXSW didn’t disappoint! Among the legions of Interactive and Film conference goers, were scores of locals partaking in the immersive spectacle of Austin–attendees in all, 50k strong in the first week. For a bay area San Francisco startup, weird is relative.

At SXSW, Bravo promotes its new show, Stripped on 6th Street in Austin, and on our right, San Francisco from my Facebook feed in the same day, #stayweird. Photo credit: thanks Shannon! (?)

Companies, marketers, creatives, tastemakers, brands, and bands are all vying for visibility, reach, and engagement. The hype game was strong; however, if you could but for a moment suspend a cynic’s disbelief, those ice-cold Lone Stars and live music erupting in every drizzling corner of Austin became magical.

UnifyID was incredibly honored to place #1 in the Security and Privacy track at SXSW’s annual Interactive Accelerator but part of what made that win so sweet was the goodwill of the SXSW attendees. In a surprising moment on stage, a room of about 400 people became an intimate family affair for a few minutes. Together, we all sang happy birthday to Sophie, John’s newly turned 4-year old daughter.

Thank you to everyone at SXSW for making our exclusive Silent Disco Brunch a success and our Accelerator Pitch one for the books!

UnifyID Scores a Unanimous Win at RSA Innovation Sandbox!

Behind every great idea, there lies a kernel of unequivocal human truth and a long road of execution to realize those intentions. On Monday, February 13th, the UnifyID team delivered and unanimously won RSA’s 2017 Innovation Sandbox competition.

“UnifyID demonstrated they were the most innovative by proving there is a way to actually leverage the individuality of humans to improve security.”
– Linda Gray Martin, Director & General Manager of RSA Conference. 

UnifyID Founder and CEO, John Whaley captivated a 1,200-person standing-room-only audience on its toes after a 3-minute pitch and 3-minute rapid-fire line of questioning from a panel of venture capitalists, entrepreneurs, and large security company judges.

Watch the 3-minute pitch below!

Many thanks to RSA and all our supporters who also saw that unequivocal human truth: there is only one you in the world.

We are on a mission to change the world and build a revolutionary identity platform based on implicit authentication to make your security seamless.

Announcing the UnifyID Spring AI Fellowship

Today, we would like to announce the UnifyID AI Fellowship program for Spring 2017. This is the second edition of the fellowship (Fall 2016 cohort) and is expected to run for 12 weeks, February 23 through May 18. This selective, cross-disciplinary program covers the following areas:

  • Deep Learning
  • Signal Processing
  • Optimization Theory
  • Sensor Technology
  • Mobile Development
  • Statistical Machine Learning
  • Security and Identity
  • Human Behavior
  • UX/UI Development for the above areas
  • Tech Journalism for the above areas
  • Special Focus:

We will be assigning one fellow to work on fakenewschallenge.org in collaboration with Dr. Dean Pomerleau of the Carnegie Mellon University Robotics Institute. If interested, please add a note in your application. We expect this fellowship applicant to have substantial experience with handling textual data and NLP expertise. The application should reflect links to previous work in this domain.

 

FELLOWSHIP DETAILS

Our UnifyID AI Fellows will be initially allocated to a well-defined project matched with their area of interest and expertise and also mapped to a fellowship mentor. The fellows are then presented with a week’s time to collaborate with the mentor and come up with an 11-week timeline roughly detailing the pathway that they plan to take to achieve the project end-goals.

During the fellowship, the fellows are expected to convene in-person and present weekly updates every Thursday evening in our office located in SoMa, San Francisco. In exceptional cases, individuals will be allowed to present via video chat. Absentees in these update-presentation sessions for two consecutive weeks will result in an automatic ejection from the fellowship.

All selected fellows will be awarded:

  1. Life-long designation as a UnifyID AI Fellow.
  2. A fellowship stipend.
  3. Access to state-of-the-art GPU hardware and $360,000 in Microsoft Azure cloud service credits.
  4. Access to our office space in SoMa.
  5. Prepaid Clipper card to help with commuting to/from the office.
  6. A chance to collaborate and publish with top-tier security experts from MIT, Stanford, CMU, Berkeley, Dartmouth, etc.
  7. Conference registration fees for all of the publications that emanate from the fellowship.
  8. Travel expenses for one flagship top-tier conference in case fellow’s work gets accepted as a publication.
  9. A citation and certificate commemorating your achievement.
  10. Exclusive UnifyID Fellow swag.
  11. A chance to present at the UnifyID Tech-expo Day in May 2017.

 

DELIVERABLES

  1. A short paper describing the project.
  2. A detailed, well-commented code submission on either ai-on.org or http://www.gitxiv.com (in case you have an arxiv worthy submission).
  3. A one-page blog post providing a less technical version of the project details. ($ ipython nbconvert–to markdown notebook.ipynb–stdout will do!)
  4. A final presentation in .ppt or .pdf format during the UnifyID Tech-expo Day.

We also expect that with regard to some of the projects, we may be able to munge certain openly available datasets and upload with associated open problems on ai-on.org if the fellow is limited by the timeline of the fellowship.

 

REQUIREMENTS

We welcome applications from practitioners, tech-enthusiasts as well as students spanning both the undergraduate and graduate levels, preferably from the SF bay area. 

 

TracksLanguagesLibraries/Platforms/Frameworks
Machine LearningPython, Lua, Julia, R, Scala, JavaScikit-learn, Torch/Autograd, Caffe, Keras with Theano/TensorFlow, Chainer
Mobile DevelopmentSwift, Objective C, JavaCore Location, Core Motion, Core Bluetooth, DeepLearningKit,Accelerate: BNNS, CoreAudio/AudioKit
SecurityC, C++, JavaScriptAES, RSA, ECDSA, PKI, Functional Encryption, Enclaves (SGX)
UX/UI Development(Portfolio Review)
Tech Journalism(Portfolio Review)

Please apply here with the following:

  1. Resume
  2. A personal statement (no longer than 250 words) explaining what you expect to achieve with this fellowship.
  3. A 5-slide presentation (ppt or pdf) detailing your most cherished accomplishment in the area you are applying to (with links to publication(s), GitHub code-base, live-project link, etc.).

 

UnifyID AI Fellowship

San Francisco, CA

Program Weekend Dates: February 23 – May 18, 2016

Application due date: January 31, 2017, 11:59 PM (PDT)

UnifyID Anoints 16 Distinguished Scientists for the AI Fellowship

Fast Growing Startup Uses Machine Learning to Solve Passwordless Authentication

Today, UnifyID, a service that can authenticate you based on unique factors like the way you walk, type, and sit, announced the final 16 fellows selected for its inaugural Artificial Intelligence Fellowship for the Fall of 2016. Each of the fellows have shown exemplary leadership and curiosity in making a meaningful difference in our society and clearly has an aptitude for making sweeping changes in this rapidly growing area of AI.

Of the company’s recent launch and success at TechCrunch Disrupt, claiming SF Battlefield Runner-Up (2nd in 1000 applicants worldwide), UnifyID CEO John Whaley said, “We were indeed overwhelmed by the amazing response to our first edition of the AI Fellowship and the sheer quality of applicants we received. We also take immense pride in the fact that more than 40% of our chosen cohort will be women, which further reinforces our commitment as one of the original 33 signees of the U.S. White House Tech Inclusion Pledge.”

The final 16 fellows hail from Israel, Paris, Kyoto, Bangalore, and cities across the U.S. with Ph.D., M.S., M.B.A., and B.S. degrees from MIT, Stanford, Berkeley, Harvard, Columbia, NYU-CIMS, UCLA, Wharton, among other top institutions.

  • Aidan Clark triple major in Math, Classical Languages and CS at UC Berkeley
  • Anna Venancio-Marques Data Scientist in Residence, PhD École normale supérieure
  • Arik Sosman Software Engineer at BitGo, 2x Apple WWDC scholar, CeBIT speaker
  • Baiyu Chen Convolutional Neural Network Researcher, Masters in CS at UC Berkeley

  • Fuxiao Xin Lead Machine Learning Scientist at GE Global Research, PhD Bioinformatics

  • Kathy Sohrabi VP Engineering, IoT and sensors, MBA at Wharton, PhD EE at UCLA
  • Kazu Komoto Chief Robotics Engineer, CNET Writer, Masters in ME at Kyoto University

  • Laura Florescu Co-authored Asymptopia, Mathematical Reviewer, PhD CS at NYU

  • Lorraine Lin Managing Director, MFE Berkeley, PhD Oxford, Masters Design Harvard
  • Morgan Lai AI Scientist, MIT Media Lab, Co-founder/CTO, M.Eng. CS at MIT
  • Pushpa Raghani Post Doc Researcher at Stanford and IBM, PhD Physics at JNCASR

  • Raul Puri Machine Learning Development at Berkeley, BS EE/CS/Bioeng at Berkeley
  • Sara Hooker Data Scientist, Founder non-profit, educational access in rural Africa
  • Siraj Raval Data Scientist, the Bill Nye of Computer Science on YouTube

  • Wentao Wang Senior New Tech Integration Engineer at Tesla, PhD ME at MIT

  • Will Grathwohl Computer Vision Specialist, Founder/Chief Scientist, BS CSAIL at MIT

 

This highly selective, cross-disciplinary program covers the following areas:

  • Deep Learning
  • Signal Processing
  • Optimization Theory
  • Sensor Technology
  • Mobile Development
  • Statistical Machine Learning
  • Security and Identity
  • Human Behavior

Our UnifyID AI Fellows will get to choose from one of 16 well-defined projects in the broad area of applied artificial intelligence in the context of solving the problem of seamless personal authentication. The Fellows will be led by our esteemed Fellowship Advisors, renown experts in machine learning and PhDs from CMU, Stanford, and University of Vienna, Austria.

Please welcome our incoming class! ✨

 

Read the original UnifyID AI Fellowship Announcement:

https://unify.id/2016/10/10/announcing-the-unifyid-ai-fellowship/

 

Initial Release:

http://www.prweb.com/releases/2016/unifyid/prweb13804371.htm#!

Announcing the UnifyID AI Fellowship

Today, we would like to announce the UnifyID AI Fellowship program for Fall 2016. The fellowship runs for six weeks, beginning October 28, 2016 through to December 4, 2016. This selective, cross-disciplinary program covers the following areas:

  • Deep Learning
  • Signal Processing
  • Optimization Theory
  • Sensor Technology
  • Mobile Development
  • Statistical Machine Learning
  • Security and Identity
  • Human Behavior

Our UnifyID AI Fellows will get to choose from one of 16 well-defined projects in the broad area of applied artificial intelligence in the context of solving the problem of seamless personal authentication.

All selected fellows will be awarded:

  1. A fellowship stipend.
  2. Access to state-of-the-art GPU hardware and $360,000 in Microsoft Azure cloud service credits.
  3. Weekend access to our office space in SoMa, as well as as-needed access on weekdays.
  4. Prepaid Clipper card to help with commuting to/from the office.
  5. Chance to collaborate and publish with top-tier security experts from MIT, Stanford, CMU, Berkeley, Dartmouth, etc.
  6. A citation, certificate, and plaque commemorating your achievement.
  7. Exclusive UnifyID Fellow signature bags and sweatshirts for the Fall 2016 inaugural class.
  8. A chance to present at the UnifyID Tech-expo Day in December 2016.

We expect the work from your Fellowship to result in either a publication (with fully open-sourced code and data repository on GitHub for reproducible research) or a patent filing.

 

REQUIREMENTS

We welcome applications from practitioners, hackers, tech-enthusiasts as well as students in full-time accredited academic programs both at the undergraduate and graduate levels, preferably from the SF bay area. An ideal candidate has both math and coding chops, but more importantly, this individual is an engineer, signal-processor, hacker, and self-proclaimed guru who is comfortable with crafting, hacking, implementing, re-implementing, and breaking Machine Learning algorithms deep, shallow or otherwise.

 TracksMachine LearningMobile Dev.
LanguagesPython, Lua, Julia, R, Scala, JavaSwift, Objective C, Java
Libraries/Platforms/FrameworksScikit-learn, Torch/Autograd, Caffe, Keras with Theano/TensorFlow, ChainerCore Location, Core Motion, Core Bluetooth, DeepLearningKit, Accelerate: BNNS, CoreAudio/AudioKit
OSUbuntu, OS X, RHEL / CentOS / Fedora, iOS, Android

Please apply here and include in the open form field, a personal statement (no longer than 250 words) explaining what you expect to achieve with this fellowship along with your favorite moment in the sun (publication, GitHub code-base, live-project link).

 

UnifyID AI Fellowship

San Francisco, CA

Program Weekend Dates: October 28 – December 4, 2016

Application due date: October 17, 2016, 11:59 PM (PDT)

Exclusive: UnifyID’s Private Beta Party

So what does it mean to launch out of stealth at TechCrunch Disrupt Battlefield? Overnight, you go from an unknown to having thousands of fervent supporters of your technology and questions spanning from how the technology works to how I can get my hands on the product (apply for private beta) to if we’re hiring (we are) to if the founders are brothers (they’re close but not related).

We’re fortunate to have the chance to share our vision of making passwords obsolete with implicit authentication and demand has been brisk. We hear you loud and clear–individuals, organizations, and companies both locally and abroad struggle with remembering those pesky passwords and few believe their current rituals are secure.

When we invited a select few to our headquarters in SoMa San Francisco for our Private Beta Party, we were so pleased that the enthusiasm had only picked up since Disrupt. This elite group of initial beta testers will pave the way for everyone to successfully authenticate online or offline and with the simplicity of being yourself.

Thanks to these amazing trailblazers!

Join the revolution today and apply for our private beta. Apologies for the wait, but we are just as amped as you are to getting the product in your hands. Thank you for joining us along this journey.

 

Photo credits: Karina Furhman

 

UnifyID Disrupts Disrupt!

We’ve never been one to shy away from making our own path, and if we hear that it’s never been done that way before, we’re eager to try. Courage is in our DNA and gives us the freedom to pursue challenge without inhibitions. We’re humbled to be named TechCrunch Disrupt Battlefield Runner-Up as we launch out of stealth, but we’re even more excited to eliminate passwords forever.

Many thanks to our supporters. We have read every tweet, Facebook post, article, article comment, and secret handshake 🙂

Photo by Oren Haskins

And for those who would like to take this disruptive journey behind Disrupt, sign up for our private beta today at Unify.ID (“Apply for Private Beta” with PTCD and why you’d like to enlist).

Photo by Oren Haskins

We encourage everyone to spread the good word on UnifyID and for technical questions, our Technical Overview is a great recap for sharing.

Photo by Oren Haskins

We are actively hiring and based in SoMa, San Francisco.

Photo by Oren Haskins

We recognize the importance of diversity in the workplace and actively celebrate what makes each individual unique.

Photo by Oren Haskins

We’re on a mission to change the world and build a revolutionary identity platform based on implicit authentication to make your security seamless.