[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-9633":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":10,"languages":10,"totalLinesOfCode":10,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":16,"stars30d":17,"stars90d":15,"forks30d":15,"starsTrendScore":18,"compositeScore":19,"rankGlobal":10,"rankLanguage":10,"license":10,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":22,"hasPages":20,"topics":23,"createdAt":10,"pushedAt":10,"updatedAt":33,"readmeContent":34,"aiSummary":35,"trendingCount":15,"starSnapshotCount":15,"syncStatus":36,"lastSyncTime":37,"discoverSource":38},9633,"awesome-mlops","visenger\u002Fawesome-mlops","visenger","A curated list of references for MLOps ","https:\u002F\u002Fml-ops.org",null,13931,2074,384,11,0,8,47,1,44.95,false,"master",true,[24,25,26,27,28,29,30,31,32],"ai","data-science","devops","engineering","federated-learning","machine-learning","ml","mlops","software-engineering","2026-06-12 02:02:10","# Awesome MLOps [![Awesome](https:\u002F\u002Fcdn.rawgit.com\u002Fsindresorhus\u002Fawesome\u002Fd7305f38d29fed78fa85652e3a63e154dd8e8829\u002Fmedia\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fsindresorhus\u002Fawesome) [![Made With Love](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FMade%20With-Love-orange.svg)](https:\u002F\u002Fgithub.com\u002Fchetanraj\u002Fawesome-github-badges) \n\n![MLOps. You Desing It. Your Train It. You Run It.](awesome-mlops-intro.png)\n\n*An awesome list of references for MLOps - Machine Learning Operations :point_right: [ml-ops.org](https:\u002F\u002Fml-ops.org\u002F)*\n\n[![ko-fi](https:\u002F\u002Fko-fi.com\u002Fimg\u002Fgithubbutton_sm.svg)](https:\u002F\u002Fko-fi.com\u002FB0B416E7UI)\n\n\n[Linkedin Dr. Larysa Visengeriyeva](https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Flarysavisenger\u002F)\n\n\n\n\n# Table of Contents\n| \u003C!-- -->                         | \u003C!-- -->                         |\n| -------------------------------- | -------------------------------- |\n| [MLOps Core](#core-mlops) | [MLOps Communities](#mlops-communities) |\n| [MLOps Books](#mlops-books) | [MLOps Articles](#mlops-articles) |\n| [MLOps Workflow Management](#wfl-management)| [MLOps: Feature Stores](#feature-stores) | \n|[MLOps: Data Engineering (DataOps)](#dataops) | [MLOps: Model Deployment and Serving](#deployment) |\n| [MLOps: Testing, Monitoring and Maintenance](#testing-monintoring)| [MLOps: Infrastructure](#mlops-infra)| \n|[MLOps Papers](#mlops-papers) | [Talks About MLOps](#talks-about-mlops) | \n| [Existing ML Systems](#existing-ml-systems) | [Machine Learning](#machine-learning)|\n| [Software Engineering](#software-engineering) | [Product Management for ML\u002FAI](#product-management-for-mlai) | \n| [The Economics of ML\u002FAI](#the-economics-of-mlai) | [Model Governance, Ethics, Responsible AI](#ml-governance) | \n| [MLOps: People & Processes](#teams)|[Newsletters About MLOps, Machine Learning, Data Science and Co.](#newsletters)| \n\n\n\u003Ca name=\"core-mlops\">\u003C\u002Fa>\n# MLOps Core\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [Machine Learning Operations: You Design It, You Train It, You Run It!](https:\u002F\u002Fml-ops.org\u002F)\n1. [MLOps SIG Specification](https:\u002F\u002Fgithub.com\u002Ftdcox\u002Fmlops-roadmap\u002Fblob\u002Fmaster\u002FMLOpsRoadmap2020.md)\n1. [ML in Production](http:\u002F\u002Fmlinproduction.com\u002F)\n1. [Awesome production machine learning: State of MLOps Tools and Frameworks](https:\u002F\u002Fgithub.com\u002FEthicalML\u002Fawesome-production-machine-learning)\n1. [Udemy “Deployment of ML Models”](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fdeployment-of-machine-learning-models\u002F)\n1. [Full Stack Deep Learning](https:\u002F\u002Fcourse.fullstackdeeplearning.com\u002F)\n1. [Engineering best practices for Machine Learning](https:\u002F\u002Fse-ml.github.io\u002Fpractices\u002F)\n1. [:rocket: Putting ML in Production](https:\u002F\u002Fmadewithml.com\u002Fcourses\u002Fputting-ml-in-production\u002F)\n1. [Stanford MLSys Seminar Series](https:\u002F\u002Fmlsys.stanford.edu\u002F)\n1. [IBM ML Operationalization Starter Kit](https:\u002F\u002Fgithub.com\u002Fibm-cloud-architecture\u002Frefarch-ml-ops)\n1. [Productize ML. A self-study guide for Developers and Product Managers building Machine Learning products.](https:\u002F\u002Fproductizeml.gitbook.io\u002Fproductize-ml\u002F)\n1. [MLOps (Machine Learning Operations) Fundamentals on GCP](https:\u002F\u002Fwww.coursera.org\u002Flearn\u002Fmlops-fundamentals)\n1. [ML full Stack preparation](https:\u002F\u002Fwww.confetti.ai\u002F)\n1. [MLOps Guide: Theory and Implementation](https:\u002F\u002Fmlops-guide.github.io\u002F)\n1. [Practitioners guide to MLOps: A framework for continuous delivery and automation of machine learning.](https:\u002F\u002Fservices.google.com\u002Ffh\u002Ffiles\u002Fmisc\u002Fpractitioners_guide_to_mlops_whitepaper.pdf)\n1. [MLOps maturity assessment](https:\u002F\u002Fgithub.com\u002Fmarvelousmlops\u002Fmlops_maturity_assessment)\n\u003C\u002Fdetails>\n\n\n\u003Ca name=\"mlops-communities\">\u003C\u002Fa>\n# MLOps Communities\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n\n1. [MLOps.community](https:\u002F\u002Fmlops.community\u002F)\n1. [CDF Special Interest Group - MLOps](https:\u002F\u002Fgithub.com\u002Fcdfoundation\u002Fsig-mlops)\n1. [RsqrdAI - Robust and Responsible AI](https:\u002F\u002Fwww.rsqrdai.org)\n1. [DataTalks.Club](https:\u002F\u002Fdatatalks.club\u002F)\n1. [Synthetic Data Community](https:\u002F\u002Fsyntheticdata.community\u002F)\n1. [MLOps World Community](https:\u002F\u002Fwww.mlopsworld.com)\n1. [Marvelous MLOps](https:\u002F\u002Fwww.linkedin.com\u002Fcompany\u002Fmarvelous-mlops)\n\u003C\u002Fdetails>\n\n\u003Ca name=\"mlops-courses\">\u003C\u002Fa>\n# MLOps Courses\n\n1. [MLOps Zoomcamp (free)](https:\u002F\u002Fgithub.com\u002FDataTalksClub\u002Fmlops-zoomcamp)\n1. [Coursera's Machine Learning Engineering for Production (MLOps) Specialization](https:\u002F\u002Fwww.coursera.org\u002Fspecializations\u002Fmachine-learning-engineering-for-production-mlops)\n1. [Udacity Machine Learning DevOps Engineer](https:\u002F\u002Fwww.udacity.com\u002Fcourse\u002Fmachine-learning-dev-ops-engineer-nanodegree--nd0821)\n1. [Made with ML](https:\u002F\u002Fmadewithml.com\u002F#course)\n1. [Udacity LLMOps: Building Real-World Applications With Large Language Models](https:\u002F\u002Fwww.udacity.com\u002Fcourse\u002Fbuilding-real-world-applications-with-large-language-models--cd13455)\n\n\n\u003Ca name=\"mlops-books\">\u003C\u002Fa>\n# MLOps Books\n\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [“Machine Learning Engineering” by Andriy Burkov, 2020](http:\u002F\u002Fwww.mlebook.com\u002Fwiki\u002Fdoku.php?id=start)\n1. [\"ML Ops: Operationalizing Data Science\" by David Sweenor, Steven Hillion, Dan Rope, Dev Kannabiran, Thomas Hill, Michael O'Connell](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fml-ops-operationalizing\u002F9781492074663\u002F)\n1. [\"Building Machine Learning Powered Applications\" by Emmanuel Ameisen](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fbuilding-machine-learning\u002F9781492045106\u002F)\n1. [\"Building Machine Learning Pipelines\" by Hannes Hapke, Catherine Nelson, 2020, O’Reilly](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fbuilding-machine-learning\u002F9781492053187\u002F) \n1. [\"Managing Data Science\" by Kirill Dubovikov](https:\u002F\u002Fwww.packtpub.com\u002Feu\u002Fdata\u002Fmanaging-data-science)\n1. [\"Accelerated DevOps with AI, ML & RPA: Non-Programmer's Guide to AIOPS & MLOPS\" by Stephen Fleming](https:\u002F\u002Fwww.amazon.com\u002FAccelerated-DevOps-AI-RPA-Non-Programmers-ebook\u002Fdp\u002FB07ZMJCJRS)\n1. [\"Evaluating Machine Learning Models\" by Alice Zheng](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fevaluating-machine-learning\u002F9781492048756\u002F)\n1. [Agile AI. 2020. By Carlo Appugliese, Paco Nathan, William S. Roberts. O'Reilly Media, Inc.](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fagile-ai\u002F9781492074984\u002F)\n1. [\"Machine Learning Logistics\". 2017. By T. Dunning et al. O'Reilly Media Inc.](https:\u002F\u002Fmapr.com\u002Febook\u002Fmachine-learning-logistics\u002F)\n1. [\"Machine Learning Design Patterns\" by Valliappa Lakshmanan, Sara Robinson, Michael Munn. O'Reilly 2020](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fmachine-learning-design\u002F9781098115777\u002F)\n1. [\"Serving Machine Learning Models: A Guide to Architecture, Stream Processing Engines, and Frameworks\" by Boris Lublinsky, O'Reilly Media, Inc. 2017](https:\u002F\u002Fwww.lightbend.com\u002Febooks\u002Fmachine-learning-guide-architecture-stream-processing-frameworks-oreilly)\n1. [\"Kubeflow for Machine Learning\" by Holden Karau, Trevor Grant, Ilan Filonenko, Richard Liu, Boris Lublinsky](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fkubeflow-for-machine\u002F9781492050117\u002F)\n1. [\"Clean Machine Learning Code\" by Moussa Taifi. Leanpub. 2020](https:\u002F\u002Fleanpub.com\u002Fcleanmachinelearningcode)\n1. [E-Book \"Practical MLOps. How to Get Ready for Production Models\"](https:\u002F\u002Fvalohai.com\u002Fmlops-ebook\u002F)\n1. [\"Introducing MLOps\" by Mark Treveil, et al. O'Reilly Media, Inc. 2020](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fintroducing-mlops\u002F9781492083283\u002F)\n1. [\"Machine Learning for Data Streams with Practical Examples in MOA\", Bifet, Albert and Gavald\\`a, Ricard and Holmes, Geoff and Pfahringer, Bernhard, MIT Press, 2018](https:\u002F\u002Fmoa.cms.waikato.ac.nz\u002Fbook\u002F)\n1. [\"Machine Learning Product Manual\" by Laszlo Sragner, Chris Kelly](https:\u002F\u002Fmachinelearningproductmanual.com\u002F)\n1. [\"Data Science Bootstrap Notes\" by Eric J. Ma](https:\u002F\u002Fericmjl.github.io\u002Fdata-science-bootstrap-notes\u002F)\n1. [\"Data Teams\" by Jesse Anderson, 2020](https:\u002F\u002Fwww.datateams.io\u002F)\n1. [\"Data Science on AWS\" by Chris Fregly, Antje Barth, 2021](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fdata-science-on\u002F9781492079385\u002F)\n1. [“Engineering MLOps” by Emmanuel Raj, 2021](https:\u002F\u002Fwww.packtpub.com\u002Fproduct\u002Fengineering-mlops\u002F9781800562882)\n1. [Machine Learning Engineering in Action](https:\u002F\u002Fwww.manning.com\u002Fbooks\u002Fmachine-learning-engineering-in-action)\n1. [Practical MLOps](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fpractical-mlops\u002F9781098103002\u002F)\n1. [\"Effective Data Science Infrastructure\" by Ville Tuulos, 2021](https:\u002F\u002Fwww.manning.com\u002Fbooks\u002Feffective-data-science-infrastructure)\n1. [AI and Machine Learning for On-Device Development, 2021, By Laurence Moroney. O'Reilly](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fai-and-machine\u002F9781098101732\u002F)\n1. [Designing Machine Learning Systems ,2022 by Chip Huyen , O'Reilly ](https:\u002F\u002Fwww.oreilly.com\u002Flibrary\u002Fview\u002Fdesigning-machine-learning\u002F9781098107956\u002F)\n1. [Reliable Machine Learning. 2022. By Cathy Chen, Niall Richard Murphy, Kranti Parisa, D. Sculley, Todd Underwood. O'Reilly](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Freliable-machine-learning\u002F9781098106218\u002F)\n1. [MLOps Lifecycle Toolkit. 2023. By Dayne Sorvisto. Apress](https:\u002F\u002Flink.springer.com\u002Fbook\u002F10.1007\u002F978-1-4842-9642-4)\n1. [Implementing MLOps in the Enterprise. 2023. By Yaron Haviv, Noah Gift. O'Reilly](https:\u002F\u002Fwww.oreilly.com\u002Flibrary\u002Fview\u002Fimplementing-mlops-in\u002F9781098136574\u002F)\n\n\u003C\u002Fdetails>\n\n\u003Ca name=\"mlops-articles\">\u003C\u002Fa>\n# MLOps Articles\n\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [Continuous Delivery for Machine Learning (by Thoughtworks)](https:\u002F\u002Fmartinfowler.com\u002Farticles\u002Fcd4ml.html)\n1. [What is MLOps? NVIDIA Blog](https:\u002F\u002Fblogs.nvidia.com\u002Fblog\u002F2020\u002F09\u002F03\u002Fwhat-is-mlops\u002F)\n1. [MLSpec: A project to standardize the intercomponent schemas for a multi-stage ML Pipeline.](https:\u002F\u002Fgithub.com\u002Fvisenger\u002FMLSpec)\n1. [The 2021 State of Enterprise Machine Learning](https:\u002F\u002Finfo.algorithmia.com\u002Ftt-state-of-ml-2021) | State of Enterprise ML 2020: [PDF](https:\u002F\u002Finfo.algorithmia.com\u002Fhubfs\u002F2019\u002FWhitepapers\u002FThe-State-of-Enterprise-ML-2020\u002FAlgorithmia_2020_State_of_Enterprise_ML.pdf) and [Interactive](https:\u002F\u002Falgorithmia.com\u002Fstate-of-ml)\n1. [Organizing machine learning projects: project management guidelines.](https:\u002F\u002Fwww.jeremyjordan.me\u002Fml-projects-guide\u002F)\n1. [Rules for ML Project (Best practices)](http:\u002F\u002Fmartin.zinkevich.org\u002Frules_of_ml\u002Frules_of_ml.pdf)\n1. [ML Pipeline Template](https:\u002F\u002Fwww.agilestacks.com\u002Ftutorials\u002Fml-pipelines)\n1. [Data Science Project Structure](https:\u002F\u002Fdrivendata.github.io\u002Fcookiecutter-data-science\u002F#directory-structure)\n1. [Reproducible ML](https:\u002F\u002Fgithub.com\u002Fcmawer\u002Freproducible-model)\n1. [ML project template facilitating both research and production phases.](https:\u002F\u002Fgithub.com\u002Fvisenger\u002Fml-project-template)\n1. [Machine learning requires a fundamentally different deployment approach. As organizations embrace machine learning, the need for new deployment tools and strategies grows.](https:\u002F\u002Fwww.oreilly.com\u002Fradar\u002Fmachine-learning-requires-a-fundamentally-different-deployment-approach\u002F)\n1. [Introducting Flyte: A Cloud Native Machine Learning and Data Processing Platform](https:\u002F\u002Feng.lyft.com\u002Fintroducing-flyte-cloud-native-machine-learning-and-data-processing-platform-fb2bb3046a59)\n1. [Why is DevOps for Machine Learning so Different?](https:\u002F\u002Fhackernoon.com\u002Fwhy-is-devops-for-machine-learning-so-different-384z32f1)\n1. [Lessons learned turning machine learning models into real products and services – O’Reilly](https:\u002F\u002Fwww.oreilly.com\u002Fradar\u002Flessons-learned-turning-machine-learning-models-into-real-products-and-services\u002F)\n1. [MLOps: Model management, deployment and monitoring with Azure Machine Learning](https:\u002F\u002Fdocs.microsoft.com\u002Fen-gb\u002Fazure\u002Fmachine-learning\u002Fconcept-model-management-and-deployment)\n1. [Guide to File Formats for Machine Learning: Columnar, Training, Inferencing, and the Feature Store](https:\u002F\u002Ftowardsdatascience.com\u002Fguide-to-file-formats-for-machine-learning-columnar-training-inferencing-and-the-feature-store-2e0c3d18d4f9)\n1. [Architecting a Machine Learning Pipeline How to build scalable Machine Learning systems](https:\u002F\u002Ftowardsdatascience.com\u002Farchitecting-a-machine-learning-pipeline-a847f094d1c7)\n1. [Why Machine Learning Models Degrade In Production](https:\u002F\u002Ftowardsdatascience.com\u002Fwhy-machine-learning-models-degrade-in-production-d0f2108e9214)\n1. [Concept Drift and Model Decay in Machine Learning](http:\u002F\u002Fxplordat.com\u002F2019\u002F04\u002F25\u002Fconcept-drift-and-model-decay-in-machine-learning\u002F?source=post_page---------------------------)\n1. [Machine Learning in Production: Why You Should Care About Data and Concept Drift](https:\u002F\u002Ftowardsdatascience.com\u002Fmachine-learning-in-production-why-you-should-care-about-data-and-concept-drift-d96d0bc907fb)\n1. [Bringing ML to Production](https:\u002F\u002Fwww.slideshare.net\u002Fmikiobraun\u002Fbringing-ml-to-production-what-is-missing-amld-2020)\n1. [A Tour of End-to-End Machine Learning Platforms](https:\u002F\u002Fdatabaseline.tech\u002Fa-tour-of-end-to-end-ml-platforms\u002F)\n1. [MLOps: Continuous delivery and automation pipelines in machine learning](https:\u002F\u002Fcloud.google.com\u002Fsolutions\u002Fmachine-learning\u002Fmlops-continuous-delivery-and-automation-pipelines-in-machine-learning)\n1. [AI meets operations](https:\u002F\u002Fwww.oreilly.com\u002Fradar\u002Fai-meets-operations\u002F)\n1. [What would machine learning look like if you mixed in DevOps? Wonder no more, we lift the lid on MLOps](https:\u002F\u002Fwww.theregister.co.uk\u002F2020\u002F03\u002F07\u002Fdevops_machine_learning_mlops\u002F)\n1. [Forbes: The Emergence Of ML Ops](https:\u002F\u002Fwww.forbes.com\u002Fsites\u002Fcognitiveworld\u002F2020\u002F03\u002F08\u002Fthe-emergence-of-ml-ops\u002F#72f04ed04698)\n1. [Cognilytica Report \"ML Model Management and Operations 2020 (MLOps)\"](https:\u002F\u002Fwww.cognilytica.com\u002F2020\u002F03\u002F03\u002Fml-model-management-and-operations-2020-mlops\u002F) \n1. [Introducing Cloud AI Platform Pipelines](https:\u002F\u002Fcloud.google.com\u002Fblog\u002Fproducts\u002Fai-machine-learning\u002Fintroducing-cloud-ai-platform-pipelines)\n1. [A Guide to Production Level Deep Learning ](https:\u002F\u002Fgithub.com\u002Falirezadir\u002FProduction-Level-Deep-Learning\u002Fblob\u002Fmaster\u002FREADME.md)\n1. [The 5 Components Towards Building Production-Ready Machine Learning Systems](https:\u002F\u002Fmedium.com\u002Fcracking-the-data-science-interview\u002Fthe-5-components-towards-building-production-ready-machine-learning-system-a4d5237ec04e)\n1. [Deep Learning in Production (references about deploying deep learning-based models in production)](https:\u002F\u002Fgithub.com\u002Fahkarami\u002FDeep-Learning-in-Production)\n1. [Machine Learning Experiment Tracking](https:\u002F\u002Ftowardsdatascience.com\u002Fmachine-learning-experiment-tracking-93b796e501b0)\n1. [The Team Data Science Process (TDSP)](https:\u002F\u002Fdocs.microsoft.com\u002Fen-us\u002Fazure\u002Fmachine-learning\u002Fteam-data-science-process\u002Foverview)\n1. [MLOps Solutions (Azure based)](https:\u002F\u002Fgithub.com\u002Fvisenger\u002FMLOps)\n1. [Monitoring ML pipelines](https:\u002F\u002Fintothedepthsofdataengineering.wordpress.com\u002F2020\u002F02\u002F13\u002Fmonitoring-ml-pipelines\u002F)\n1. [Deployment & Explainability of Machine Learning COVID-19 Solutions at Scale with Seldon Core and Alibi](https:\u002F\u002Fgithub.com\u002Faxsaucedo\u002Fseldon-core\u002Ftree\u002Fcorona_research_exploration\u002Fexamples\u002Fmodels\u002Fresearch_paper_classification)\n1. [Demystifying AI Infrastructure](https:\u002F\u002Fwww.intel.com\u002Fcontent\u002Fwww\u002Fus\u002Fen\u002Fintel-capital\u002Fnews\u002Fstory.html?id=a0F1I00000BNTXPUA5#\u002Ftype=All\u002Fpage=0\u002Fterm=\u002Ftags=)\n1. [Organizing machine learning projects: project management guidelines.](https:\u002F\u002Fwww.jeremyjordan.me\u002Fml-projects-guide\u002F)\n1. [The Checklist for Machine Learning Projects (from Aurélien Géron,\"Hands-On Machine Learning with Scikit-Learn and TensorFlow\")](https:\u002F\u002Fgithub.com\u002Fvisenger\u002Fhandson-ml\u002Fblob\u002Fmaster\u002Fml-project-checklist.md)\n1. [Data Project Checklist by Jeremy Howard](https:\u002F\u002Fwww.fast.ai\u002F2020\u002F01\u002F07\u002Fdata-questionnaire\u002F)\n1. [MLOps: not as Boring as it Sounds](https:\u002F\u002Fitnext.io\u002Fmlops-not-as-boring-as-it-sounds-eaebe73e3533)\n1. [10 Steps to Making Machine Learning Operational. Cloudera White Paper](https:\u002F\u002Fwww.cloudera.com\u002Fcontent\u002Fdam\u002Fwww\u002Fmarketing\u002Fresources\u002Fwhitepapers\u002F10-steps-to-making-ml-operational.pdf)\n1. [MLOps is Not Enough. The Need for an End-to-End Data Science Lifecycle Process.](https:\u002F\u002Ftechcommunity.microsoft.com\u002Ft5\u002Fazure-ai\u002Fmlops-is-not-enough\u002Fba-p\u002F1386789)\n1. [Data Science Lifecycle Repository Template](https:\u002F\u002Fgithub.com\u002Fdslp\u002Fdslp-repo-template)\n1. [Template: code and pipeline definition for a machine learning project demonstrating how to automate an end to end ML\u002FAI workflow. ](https:\u002F\u002Fgithub.com\u002Faronchick\u002FMLOps-pipeline)\n1. [Nitpicking Machine Learning Technical Debt](https:\u002F\u002Fmatthewmcateer.me\u002Fblog\u002Fmachine-learning-technical-debt\u002F)\n1. [The Best Tools, Libraries, Frameworks and Methodologies that Machine Learning Teams Actually Use – Things We Learned from 41 ML Startups](https:\u002F\u002Fneptune.ai\u002Fblog\u002Ftools-libraries-frameworks-methodologies-ml-startups-roundup)\n1. [Software Engineering for AI\u002FML - An Annotated Bibliography](https:\u002F\u002Fgithub.com\u002Fckaestne\u002Fseaibib)\n1. [Intelligent System. Machine Learning in Practice](https:\u002F\u002Fintelligentsystem.io\u002F)\n1. [CMU 17-445\u002F645: Software Engineering for AI-Enabled Systems (SE4AI)](https:\u002F\u002Fgithub.com\u002Fckaestne\u002Fseai\u002F)\n1. [Machine Learning is Requirements Engineering](https:\u002F\u002Flink.medium.com\u002Fl7akzjR826)\n1. [Machine Learning Reproducibility Checklist](https:\u002F\u002Fwww.cs.mcgill.ca\u002F~jpineau\u002FReproducibilityChecklist.pdf)\n1. [Machine Learning Ops. A collection of resources on how to facilitate Machine Learning Ops with GitHub.](http:\u002F\u002Fmlops-github.com\u002F)\n1. [Task Cheatsheet for Almost Every Machine Learning Project A checklist of tasks for building End-to-End ML projects](https:\u002F\u002Ftowardsdatascience.com\u002Ftask-cheatsheet-for-almost-every-machine-learning-project-d0946861c6d0)\n1. [Web services vs. streaming for real-time machine learning endpoints](https:\u002F\u002Ftowardsdatascience.com\u002Fweb-services-vs-streaming-for-real-time-machine-learning-endpoints-c08054e2b18e)\n1. [How PyTorch Lightning became the first ML framework to run continuous integration on TPUs](https:\u002F\u002Fmedium.com\u002Fpytorch\u002Fhow-pytorch-lightning-became-the-first-ml-framework-to-runs-continuous-integration-on-tpus-a47a882b2c95)\n1. [The ultimate guide to building maintainable Machine Learning pipelines using DVC](https:\u002F\u002Ftowardsdatascience.com\u002Fthe-ultimate-guide-to-building-maintainable-machine-learning-pipelines-using-dvc-a976907b2a1b)\n1. [Continuous Machine Learning (CML) is CI\u002FCD for Machine Learning Projects (DVC)](https:\u002F\u002Fcml.dev\u002F)\n1. [What I learned from looking at 200 machine learning tools](https:\u002F\u002Fhuyenchip.com\u002F2020\u002F06\u002F22\u002Fmlops.html) | Update: [MLOps Tooling Landscape v2 (+84 new tools) - Dec '20](https:\u002F\u002Fdocs.google.com\u002Fspreadsheets\u002Fd\u002F10pPQYmyNnYb6zshOKxBjJ704E0XUj2vJ9HCDfoZxAoA\u002Fedit#gid=1651929178)\n1. [Big Data & AI Landscape](http:\u002F\u002Fmattturck.com\u002Fwp-content\u002Fuploads\u002F2018\u002F07\u002FMatt_Turck_FirstMark_Big_Data_Landscape_2018_Final.png)\n1. [Deploying Machine Learning Models as Data, not Code — A better match?](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploying-machine-learning-models-as-data-not-code-omega-ml-8825a0ae530a)\n1. [“Thou shalt always scale” — 10 commandments of MLOps](https:\u002F\u002Ftowardsdatascience.com\u002Fmlops-thou-shalt-always-scale-10-commandments-of-mlops-152c11e711a5)\n1. [Three Risks in Building Machine Learning Systems](https:\u002F\u002Finsights.sei.cmu.edu\u002Fsei_blog\u002F2020\u002F05\u002Fthree-risks-in-building-machine-learning-systems.html)\n1. [Blog about ML in production (by maiot.io)](https:\u002F\u002Fblog.maiot.io\u002F)\n1. Back to the Machine Learning fundamentals: How to write code for Model deployment. [Part 1](https:\u002F\u002Fmedium.com\u002F@ivannardini\u002Fback-to-the-machine-learning-fundamentals-how-to-write-code-for-model-deployment-part-1-3-4b05deda1cd1), [Part 2](https:\u002F\u002Fmedium.com\u002F@ivannardini\u002Fback-to-the-machine-learning-fundamentals-how-to-write-code-for-model-deployment-part-2-3-9632d5a43f98), [Part 3](https:\u002F\u002Fmedium.com\u002F@ivannardini\u002Fback-to-the-machine-learning-fundamentals-how-to-write-code-for-model-deployment-part-3-3-fb85102bebb2)\n1. [MLOps: Machine Learning as an Engineering Discipline](https:\u002F\u002Ftowardsdatascience.com\u002Fml-ops-machine-learning-as-an-engineering-discipline-b86ca4874a3f)\n1. [ML Engineering on Google Cloud Platform (hands-on labs and code samples)](https:\u002F\u002Fgithub.com\u002FGoogleCloudPlatform\u002Fmlops-on-gcp)\n1. [Deep Reinforcement Learning in Production. The use of Reinforcement Learning to Personalize User Experience at Zynga](https:\u002F\u002Ftowardsdatascience.com\u002Fdeep-reinforcement-learning-in-production-7e1e63471e2)\n1. [What is Data Observability?](https:\u002F\u002Ftowardsdatascience.com\u002Fwhat-is-data-observability-40b337971e3e)\n1. [A Practical Guide to Maintaining Machine Learning in Production](https:\u002F\u002Feugeneyan.com\u002Fwriting\u002Fpractical-guide-to-maintaining-machine-learning\u002F)\n1. Continuous Machine Learning. [Part 1](https:\u002F\u002Fmribeirodantas.xyz\u002Fblog\u002Findex.php\u002F2020\u002F08\u002F10\u002Fcontinuous-machine-learning\u002F), [Part 2](https:\u002F\u002Fmribeirodantas.xyz\u002Fblog\u002Findex.php\u002F2020\u002F08\u002F18\u002Fcontinuous-machine-learning-part-ii\u002F). Part 3 is coming soon.\n1. [The Agile approach in data science explained by an ML expert](https:\u002F\u002Fwww.iunera.com\u002Fkraken\u002Fbig-data-science-strategy\u002Fthe-agile-approach-in-data-science-explained-by-an-ml-expert\u002F)\n1. [Here is what you need to look for in a model server to build ML-powered services](https:\u002F\u002Fanyscale.com\u002Fblog\u002Fheres-what-you-need-to-look-for-in-a-model-server-to-build-ml-powered-services\u002F)\n1. [The problem with AI developer tools for enterprises (and what IKEA has to do with it)](https:\u002F\u002Ftowardsdatascience.com\u002Fthe-problem-with-ai-developer-tools-for-enterprises-and-what-ikea-has-to-do-with-it-b26277841661)\n1. [Streaming Machine Learning with Tiered Storage](https:\u002F\u002Fwww.confluent.io\u002Fblog\u002Fstreaming-machine-learning-with-tiered-storage\u002F)\n1. [Best practices for performance and cost optimization for machine learning (Google Cloud)](https:\u002F\u002Fcloud.google.com\u002Fsolutions\u002Fmachine-learning\u002Fbest-practices-for-ml-performance-cost)\n1. [Lean Data and Machine Learning Operations](https:\u002F\u002Fdatabaseline.tech\u002Flean-dml-operations\u002F)\n1. [A Brief Guide to Running ML Systems in Production Best Practices for Site Reliability Engineers](https:\u002F\u002Fwww.oreilly.com\u002Fcontent\u002Fa-brief-guide-to-running-ml-systems-in-production\u002F)\n1. [AI engineering practices in the wild - SIG | Getting software right for a healthier digital world](https:\u002F\u002Fwww.softwareimprovementgroup.com\u002Fresources\u002Fai-engineering-practices-in-the-wild\u002F)\n1. [SE-ML | The 2020 State of Engineering Practices for Machine Learning](https:\u002F\u002Fse-ml.github.io\u002Freport2020)\n1. [Awesome Software Engineering for Machine Learning (GitHub repository)](https:\u002F\u002Fgithub.com\u002FSE-ML\u002Fawesome-seml)\n1. [Sampling isn’t enough, profile your ML data instead](https:\u002F\u002Ftowardsdatascience.com\u002Fsampling-isnt-enough-profile-your-ml-data-instead-6a28fcfb2bd4?source=friends_link&sk=5af46143562d348b182c449265ed54fb)\n1. [Reproducibility in ML: why it matters and how to achieve it](https:\u002F\u002Fdetermined.ai\u002Fblog\u002Freproducibility-in-ml\u002F)\n1. [12 Factors of reproducible Machine Learning in production](https:\u002F\u002Fblog.maiot.io\u002F12-factors-of-ml-in-production\u002F)\n1. [MLOps: More Than Automation](https:\u002F\u002Fdevops.com\u002Fmlop-more-than-automation\u002F)\n1. [Lean Data Science](https:\u002F\u002Flocallyoptimistic.com\u002Fpost\u002Flean-data-science\u002F)\n1. [Engineering Skills for Data Scientists](https:\u002F\u002Fmark.douthwaite.io\u002Ftag\u002Fengineering-skills-for-data-scientists\u002F)\n1. [DAGsHub Blog. Read about data science and machine learning workflows, MLOps, and open source data science](https:\u002F\u002Fdagshub.com\u002Fblog\u002F)\n1. [Data Science Project Flow for Startups](https:\u002F\u002Ftowardsdatascience.com\u002Fdata-science-project-flow-for-startups-282a93d4508d)\n1. [Data Science Engineering at Shopify](https:\u002F\u002Fshopify.engineering\u002Ftopics\u002Fdata-science-engineering)\n1. [Building state-of-the-art machine learning technology with efficient execution for the crypto economy](https:\u002F\u002Fblog.coinbase.com\u002Fbuilding-state-of-the-art-machine-learning-technology-with-efficient-execution-for-the-crypto-ad10896a48a)\n1. [Completing the Machine Learning Loop](https:\u002F\u002Fjimmymwhitaker.medium.com\u002Fcompleting-the-machine-learning-loop-e03c784eaab4)\n1. [Deploying Machine Learning Models: A Checklist](https:\u002F\u002Ftwolodzko.github.io\u002Fml-checklist) \n1. [Global MLOps and ML tools landscape (by MLReef)](https:\u002F\u002Fabout.mlreef.com\u002Fblog\u002Fglobal-mlops-and-ml-tools-landscape)\n1. [Why all Data Science teams need to get serious about MLOps](https:\u002F\u002Ftowardsdatascience.com\u002Fwhy-data-science-teams-needs-to-get-serious-about-mlops-56c98e255e20)  \n1. [MLOps Values (by Bart Grasza)](https:\u002F\u002Fgist.github.com\u002Fbartgras\u002F4ab9c716167b5d9aee6a222f7301ac60)\n1. [Machine Learning Systems Design (by Chip Huyen)](https:\u002F\u002Fhuyenchip.com\u002Fmachine-learning-systems-design\u002Ftoc.html)\n1. [Designing an ML system (Stanford | CS 329 | Chip Huyen)](https:\u002F\u002Fdocs.google.com\u002Fpresentation\u002Fd\u002F13a5B2HeK9Id59zy3oNJDv5_ksDvzbGmNLx4zumkimZM\u002Fedit?usp=sharing)\n1. [How COVID-19 Has Infected AI Models (about the data drift or model drift concept)](https:\u002F\u002Fwww.dominodatalab.com\u002Fblog\u002Fhow-covid-19-has-infected-ai-models\u002F)\n1. [Microkernel Architecture for Machine Learning Library. An Example of Microkernel Architecture with Python Metaclass](https:\u002F\u002Ftowardsdatascience.com\u002Fmicrokernel-architecture-for-machine-learning-library-c04b797e0d5f)\n1. [Machine Learning in production: the Booking.com approach](https:\u002F\u002Fbooking.ai\u002Fhttps-booking-ai-machine-learning-production-3ee8fe943c70)\n1. [What I Learned From Attending TWIMLcon 2021 (by James Le)](https:\u002F\u002Fjameskle.com\u002Fwrites\u002Ftwiml2021)\n1. [Designing ML Orchestration Systems for Startups. A case study in building a lightweight production-grade ML orchestration system](https:\u002F\u002Ftowardsdatascience.com\u002Fdesigning-ml-orchestration-systems-for-startups-202e527d7897)\n1. [Towards MLOps: Technical capabilities of a Machine Learning platform | Prosus AI Tech Blog](https:\u002F\u002Fmedium.com\u002Fprosus-ai-tech-blog\u002Ftowards-mlops-technical-capabilities-of-a-machine-learning-platform-61f504e3e281)\n1. [Get started with MLOps A comprehensive MLOps tutorial with open source tools](https:\u002F\u002Ftowardsdatascience.com\u002Fget-started-with-mlops-fd7062cab018)\n1. [From DevOps to MLOPS: Integrate Machine Learning Models using Jenkins and Docker](https:\u002F\u002Ftowardsdatascience.com\u002Ffrom-devops-to-mlops-integrate-machine-learning-models-using-jenkins-and-docker-79034dbedf1)\n1. [Example code for a basic ML Platform based on Pulumi, FastAPI, DVC, MLFlow and more](https:\u002F\u002Fgithub.com\u002Faporia-ai\u002Fmlplatform-workshop)\n1. [Software Engineering for Machine Learning: Characterizing and Detecting Mismatch in Machine-Learning Systems](https:\u002F\u002Finsights.sei.cmu.edu\u002Fblog\u002Fsoftware-engineering-for-machine-learning-characterizing-and-detecting-mismatch-in-machine-learning-systems\u002F)\n1. [TWIML Solutions Guide](https:\u002F\u002Ftwimlai.com\u002Fsolutions\u002Fintroducing-twiml-ml-ai-solutions-guide\u002F)\n1. [How Well Do You Leverage Machine Learning at Scale? Six Questions to Ask](https:\u002F\u002Fmedium.com\u002Fcognizantai\u002Fhow-well-do-you-leverage-machine-learning-at-scale-six-questions-to-ask-7e6acda15ea5)\n1. [Getting started with MLOps: Selecting the right capabilities for your use case](https:\u002F\u002Fcloud.google.com\u002Fblog\u002Fproducts\u002Fai-machine-learning\u002Fselect-the-right-mlops-capabilities-for-your-ml-use-case)\n1. [The Latest Work from the SEI: Artificial Intelligence, DevSecOps, and Security Incident Response](https:\u002F\u002Finsights.sei.cmu.edu\u002Fblog\u002Fthe-latest-work-from-the-sei-artificial-intelligence-devsecops-and-security-incident-response\u002F)\n1. [MLOps: The Ultimate Guide. A handbook on MLOps and how to think about it](https:\u002F\u002Ftowardsdatascience.com\u002Fmlops-the-ultimate-guide-9d902c752fd1)\n1. [Enterprise Readiness of Cloud MLOps](https:\u002F\u002Fgigaom.com\u002Freport\u002Fenterprise-readiness-of-cloud-mlops\u002F)\n1. [Should I Train a Model for Each Customer or Use One Model for All of My Customers?](https:\u002F\u002Ftowardsdatascience.com\u002Fshould-i-train-a-model-for-each-customer-or-use-one-model-for-all-of-my-customers-f9e8734d991)\n1. [MLOps-Basics (GitHub repo)](https:\u002F\u002Fgithub.com\u002Fgraviraja\u002FMLOps-Basics) by [raviraja](https:\u002F\u002Fgithub.com\u002Fgraviraja)\n1. [Another tool won’t fix your MLOps problems](https:\u002F\u002Fdshersh.medium.com\u002Ftoo-many-mlops-tools-c590430ba81b)\n1. [Best MLOps Tools: What to Look for and How to Evaluate Them (by NimbleBox.ai)](https:\u002F\u002Fnimblebox.ai\u002Fblog\u002Fmlops-tools)\n1. [MLOps vs. DevOps: A Detailed Comparison (by NimbleBox.ai)](https:\u002F\u002Fnimblebox.ai\u002Fblog\u002Fmlops-vs-devops)\n1. [A Guide To Setting Up Your MLOps Team (by NimbleBox.ai)](https:\u002F\u002Fnimblebox.ai\u002Fblog\u002Fmlops-team-structure)\n\u003C\u002Fdetails>\n\n\n\n\u003Ca name=\"wfl-management\">\u003C\u002Fa>\n# MLOps: Workflow Management\n\n1. [Open-source Workflow Management Tools: A Survey by Ploomber](https:\u002F\u002Fploomber.io\u002Fposts\u002Fsurvey\u002F)\n1. [How to Compare ML Experiment Tracking Tools to Fit Your Data Science Workflow (by dagshub)](https:\u002F\u002Fdagshub.com\u002Fblog\u002Fhow-to-compare-ml-experiment-tracking-tools-to-fit-your-data-science-workflow\u002F)\n1. [15 Best Tools for Tracking Machine Learning Experiments](https:\u002F\u002Fmedium.com\u002Fneptune-ai\u002F15-best-tools-for-tracking-machine-learning-experiments-64c6eff16808)\n\n\u003Ca name=\"feature-stores\">\u003C\u002Fa>\n# MLOps: Feature Stores\n\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [Feature Stores for Machine Learning Medium Blog](https:\u002F\u002Fmedium.com\u002Fdata-for-ai)\n1. [MLOps with a Feature Store](https:\u002F\u002Fwww.logicalclocks.com\u002Fblog\u002Fmlops-with-a-feature-store)\n1. [Feature Stores for ML](http:\u002F\u002Ffeaturestore.org\u002F)\n1. [Hopsworks: Data-Intensive AI with a Feature Store](https:\u002F\u002Fgithub.com\u002Flogicalclocks\u002Fhopsworks)\n1. [Feast: An open-source Feature Store for Machine Learning](https:\u002F\u002Fgithub.com\u002Ffeast-dev\u002Ffeast)\n1. [What is a Feature Store?](https:\u002F\u002Fwww.tecton.ai\u002Fblog\u002Fwhat-is-a-feature-store\u002F)\n1. [ML Feature Stores: A Casual Tour](https:\u002F\u002Fmedium.com\u002F@farmi\u002Fml-feature-stores-a-casual-tour-fc45a25b446a)\n1. [Comprehensive List of Feature Store Architectures for Data Scientists and Big Data Professionals](https:\u002F\u002Fhackernoon.com\u002Fthe-essential-architectures-for-every-data-scientist-and-big-data-engineer-f21u3e5c)\n1. [ML Engineer Guide: Feature Store vs Data Warehouse (vendor blog)](https:\u002F\u002Fwww.logicalclocks.com\u002Fblog\u002Ffeature-store-vs-data-warehouse)\n1. [Building a Gigascale ML Feature Store with Redis, Binary Serialization, String Hashing, and Compression (DoorDash blog)](https:\u002F\u002Fdoordash.engineering\u002F2020\u002F11\u002F19\u002Fbuilding-a-gigascale-ml-feature-store-with-redis\u002F)\n1. [Feature Stores: Variety of benefits for Enterprise AI.](https:\u002F\u002Finsidebigdata.com\u002F2020\u002F12\u002F29\u002Fhow-feature-stores-will-revolutionize-enterprise-ai\u002F)\n1. [Feature Store as a Foundation for Machine Learning](https:\u002F\u002Ftowardsdatascience.com\u002Ffeature-store-as-a-foundation-for-machine-learning-d010fc6eb2f3)\n1. [ML Feature Serving Infrastructure at Lyft](https:\u002F\u002Feng.lyft.com\u002Fml-feature-serving-infrastructure-at-lyft-d30bf2d3c32a)\n1. [Feature Stores for Self-Service Machine Learning](https:\u002F\u002Fwww.ethanrosenthal.com\u002F2021\u002F02\u002F03\u002Ffeature-stores-self-service\u002F)\n1. [The Architecture Used at LinkedIn to Improve Feature Management in Machine Learning Models.](https:\u002F\u002Fjrodthoughts.medium.com\u002Fthe-architecture-used-at-linkedin-to-improve-feature-management-in-machine-learning-models-c7bd6ae54db)\n1. [Is There a Feature Store Over the Rainbow? How to select the right feature store for your use case](https:\u002F\u002Ftowardsdatascience.com\u002Fis-there-a-feature-store-over-the-rainbow-291cab94e8a5)\n\u003C\u002Fdetails>\n \n\u003Ca name=\"dataops\">\u003C\u002Fa>\n# MLOps: Data Engineering (DataOps)\n\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [The state of data quality in 2020 – O’Reilly](https:\u002F\u002Fwww.oreilly.com\u002Fradar\u002Fthe-state-of-data-quality-in-2020\u002F)\n1. [Why We Need DevOps for ML Data](https:\u002F\u002Ftecton.ai\u002Fblog\u002Fdevops-ml-data\u002F) \n1. [Data Preparation for Machine Learning (7-Day Mini-Course)](https:\u002F\u002Fmachinelearningmastery.com\u002Fdata-preparation-for-machine-learning-7-day-mini-course\u002F)\n1. [Best practices in data cleaning: A Complete Guide to Everything You Need to Do Before and After Collecting Your Data.](https:\u002F\u002Fwww.researchgate.net\u002Fpublication\u002F266714997_Best_practices_in_data_cleaning_A_Complete_Guide_to_Everything_You_Need_to_Do_Before_and_After_Collecting_Your_Data)\n1. [17 Strategies for Dealing with Data, Big Data, and Even Bigger Data](https:\u002F\u002Ftowardsdatascience.com\u002F17-strategies-for-dealing-with-data-big-data-and-even-bigger-data-283426c7d260)\n1. [DataOps Data Architecture](https:\u002F\u002Fblog.datakitchen.io\u002Fblog\u002Fdataops-data-architecture)\n1. [Data Orchestration — A Primer](https:\u002F\u002Fmedium.com\u002Fmemory-leak\u002Fdata-orchestration-a-primer-56f3ddbb1700)\n1. [4 Data Trends to Watch in 2020](https:\u002F\u002Fmedium.com\u002Fmemory-leak\u002F4-data-trends-to-watch-in-2020-491707902c09)\n1. [CSE 291D \u002F 234: Data Systems for Machine Learning](http:\u002F\u002Fcseweb.ucsd.edu\u002Fclasses\u002Ffa20\u002Fcse291-d\u002Findex.html)\n1. [A complete picture of the modern data engineering landscape](https:\u002F\u002Fgithub.com\u002Fdatastacktv\u002Fdata-engineer-roadmap)\n1. [Continuous Integration for your data with GitHub Actions and Great Expectations. One step closer to CI\u002FCD for your data pipelines](https:\u002F\u002Fgreatexpectations.io\u002Fblog\u002Fgithub-actions\u002F)\n1. [Emerging Architectures for Modern Data Infrastructure](https:\u002F\u002Fa16z.com\u002F2020\u002F10\u002F15\u002Fthe-emerging-architectures-for-modern-data-infrastructure\u002F)\n1. [Awesome Data Engineering. Learning path and resources to become a data engineer](https:\u002F\u002Fawesomedataengineering.com\u002F)\n1. Data Quality at Airbnb [Part 1](https:\u002F\u002Fmedium.com\u002Fairbnb-engineering\u002Fdata-quality-at-airbnb-e582465f3ef7) | [Part 2](https:\u002F\u002Fmedium.com\u002Fairbnb-engineering\u002Fdata-quality-at-airbnb-870d03080469)\n1. [DataHub: Popular metadata architectures explained](https:\u002F\u002Fengineering.linkedin.com\u002Fblog\u002F2020\u002Fdatahub-popular-metadata-architectures-explained)\n1. [Financial Times Data Platform: From zero to hero. An in-depth walkthrough of the evolution of our Data Platform](https:\u002F\u002Fmedium.com\u002Fft-product-technology\u002Ffinancial-times-data-platform-from-zero-to-hero-143156bffb1d)\n1. [Alki, or how we learned to stop worrying and love cold metadata (Dropbox)](https:\u002F\u002Fdropbox.tech\u002Finfrastructure\u002Falki--or-how-we-learned-to-stop-worrying-and-love-cold-metadata)\n1. [A Beginner's Guide to Clean Data. Practical advice to spot and avoid data quality problems (by Benjamin Greve)](https:\u002F\u002Fb-greve.gitbook.io\u002Fbeginners-guide-to-clean-data\u002F)\n1. [ML Lake: Building Salesforce’s Data Platform for Machine Learning](https:\u002F\u002Fengineering.salesforce.com\u002Fml-lake-building-salesforces-data-platform-for-machine-learning-228c30e21f16)\n1. [Data Catalog 3.0: Modern Metadata for the Modern Data Stack](https:\u002F\u002Ftowardsdatascience.com\u002Fdata-catalog-3-0-modern-metadata-for-the-modern-data-stack-ec621f593dcf)\n1. [Metadata Management Systems](https:\u002F\u002Fgradientflow.com\u002Fthe-growing-importance-of-metadata-management-systems\u002F)\n1. [Essential resources for data engineers (a curated recommended read and watch list for scalable data processing)](https:\u002F\u002Fwww.scling.com\u002Freading-list\u002F)\n1. [Comprehensive and Comprehensible Data Catalogs: The What, Who, Where, When, Why, and How of Metadata Management (Paper)](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2103.07532.pdf)\n1. [What I Learned From Attending DataOps Unleashed 2021 (byJames Le)](https:\u002F\u002Fjameskle.com\u002Fwrites\u002Fdataops-unleashed2021)\n1. [Uber's Journey Toward Better Data Culture From First Principles](https:\u002F\u002Fubr.to\u002F3lo9GU8)\n1. [Cerberus - lightweight and extensible data validation library for Python](https:\u002F\u002Fdocs.python-cerberus.org\u002Fen\u002Fstable\u002F)\n1. [Design a data mesh architecture using AWS Lake Formation and AWS Glue. AWS Big Data Blog](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fbig-data\u002Fdesign-a-data-mesh-architecture-using-aws-lake-formation-and-aws-glue\u002F)\n1. [Data Management Challenges in Production Machine Learning (slides)](https:\u002F\u002Fstatic.googleusercontent.com\u002Fmedia\u002Fresearch.google.com\u002Fen\u002F\u002Fpubs\u002Farchive\u002F46178.pdf)\n1. [The Missing Piece of Data Discovery and Observability Platforms: Open Standard for Metadata](https:\u002F\u002Ftowardsdatascience.com\u002Fthe-missing-piece-of-data-discovery-and-observability-platforms-open-standard-for-metadata-37dac2d0503)\n1. [Automating Data Protection at Scale](https:\u002F\u002Fmedium.com\u002Fairbnb-engineering\u002Fautomating-data-protection-at-scale-part-1-c74909328e08)\n1. [A curated list of awesome pipeline toolkits](https:\u002F\u002Fgithub.com\u002Fpditommaso\u002Fawesome-pipeline)\n1. [Data Mesh Archtitecture](https:\u002F\u002Fwww.datamesh-architecture.com\u002F)\n1. [The Essential Guide to Data Exploration in Machine Learning (by NimbleBox.ai)](https:\u002F\u002Fnimblebox.ai\u002Fblog\u002Fdata-exploration)\n1. [Finding millions of label errors with Cleanlab](https:\u002F\u002Fdatacentricai.org\u002Fblog\u002Ffinding-millions-of-label-errors-with-cleanlab\u002F)\n\u003C\u002Fdetails>\n\n\n\n\u003Ca name=\"deployment\">\u003C\u002Fa> \n# MLOps: Model Deployment and Serving\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [AI Infrastructure for Everyone: DeterminedAI](https:\u002F\u002Fdetermined.ai\u002F)\n1. [Deploying R Models with MLflow and Docker](https:\u002F\u002Fmdneuzerling.com\u002Fpost\u002Fdeploying-r-models-with-mlflow-and-docker\u002F)\n1. [What Does it Mean to Deploy a Machine Learning Model?](https:\u002F\u002Fmlinproduction.com\u002Fwhat-does-it-mean-to-deploy-a-machine-learning-model-deployment-series-01\u002F)\n1. [Software Interfaces for Machine Learning Deployment](https:\u002F\u002Fmlinproduction.com\u002Fsoftware-interfaces-for-machine-learning-deployment-deployment-series-02\u002F)\n1. [Batch Inference for Machine Learning Deployment](https:\u002F\u002Fmlinproduction.com\u002Fbatch-inference-for-machine-learning-deployment-deployment-series-03\u002F)\n1. [AWS Cost Optimization for ML Infrastructure - EC2 spend](https:\u002F\u002Fblog.floydhub.com\u002Faws-cost-optimization-for-ml-infra-ec2\u002F)\n1. [CI\u002FCD for Machine Learning & AI](https:\u002F\u002Fblog.paperspace.com\u002Fci-cd-for-machine-learning-ai\u002F)\n1. [Itaú Unibanco: How we built a CI\u002FCD Pipeline for machine learning with ***online training*** in Kubeflow](https:\u002F\u002Fcloud.google.com\u002Fblog\u002Fproducts\u002Fai-machine-learning\u002Fitau-unibanco-how-we-built-a-cicd-pipeline-for-machine-learning-with-online-training-in-kubeflow)\n1. [101 For Serving ML Models](https:\u002F\u002Fpakodas.substack.com\u002Fp\u002F101-for-serving-ml-models-10217c9f0764)\n1. [Deploying Machine Learning models to production — **Inference service architecture patterns**](https:\u002F\u002Fmedium.com\u002Fdata-for-ai\u002Fdeploying-machine-learning-models-to-production-inference-service-architecture-patterns-bc8051f70080)\n1. [Serverless ML: Deploying Lightweight Models at Scale](https:\u002F\u002Fmark.douthwaite.io\u002Fserverless-machine-learning\u002F)\n1. ML Model Rollout To Production. [Part 1](https:\u002F\u002Fwww.superwise.ai\u002Fresources-old\u002Fsafely-rolling-out-ml-models-to-production) | [Part 2](https:\u002F\u002Fwww.superwise.ai\u002Fblog\u002Fpart-ii-safely-rolling-out-models-to-production)\n1. [Deploying Python ML Models with Flask, Docker and Kubernetes](https:\u002F\u002Falexioannides.com\u002F2019\u002F01\u002F10\u002Fdeploying-python-ml-models-with-flask-docker-and-kubernetes\u002F)\n1. [Deploying Python ML Models with Bodywork](https:\u002F\u002Falexioannides.com\u002F2020\u002F12\u002F01\u002Fdeploying-ml-models-with-bodywork\u002F)\n1. [Framework for a successful Continuous Training Strategy. When should the model be retrained? What data should be used? What should be retrained? A data-driven approach](https:\u002F\u002Ftowardsdatascience.com\u002Fframework-for-a-successful-continuous-training-strategy-8c83d17bb9dc)\n1. [Efficient Machine Learning Inference. The benefits of multi-model serving where latency matters](https:\u002F\u002Fwww.oreilly.com\u002Fcontent\u002Fefficient-machine-learning-inference\u002F)\n1. [Deploying Hugging Face ML Models in the Cloud with Infrastructure as Code](https:\u002F\u002Fwww.pulumi.com\u002Fblog\u002Fmlops-the-ai-challenge-is-cloud-not-code\u002F)\n\u003C\u002Fdetails>\n\n \n\u003Ca name=\"testing-monintoring\">\u003C\u002Fa> \n# MLOps: Testing, Monitoring and Maintenance\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [Building dashboards for operational visibility (AWS)](https:\u002F\u002Faws.amazon.com\u002Fbuilders-library\u002Fbuilding-dashboards-for-operational-visibility\u002F)\n1. [Monitoring Machine Learning Models in Production](https:\u002F\u002Fchristophergs.com\u002Fmachine%20learning\u002F2020\u002F03\u002F14\u002Fhow-to-monitor-machine-learning-models\u002F)\n1. [Effective testing for machine learning systems](https:\u002F\u002Fwww.jeremyjordan.me\u002Ftesting-ml\u002F)\n1. [Unit Testing Data: What is it and how do you do it?](https:\u002F\u002Fwinderresearch.com\u002Funit-testing-data-what-is-it-and-how-do-you-do-it\u002F)\n1. [How to Test Machine Learning Code and Systems](https:\u002F\u002Feugeneyan.com\u002Fwriting\u002Ftesting-ml\u002F) ([Accompanying code](https:\u002F\u002Fgithub.com\u002Feugeneyan\u002Ftesting-ml))\n1. [Wu, T., Dong, Y., Dong, Z., Singa, A., Chen, X. and Zhang, Y., 2020. Testing Artificial Intelligence System Towards Safety and Robustness: State of the Art. IAENG International Journal of Computer Science, 47(3).](http:\u002F\u002Fwww.iaeng.org\u002FIJCS\u002Fissues_v47\u002Fissue_3\u002FIJCS_47_3_13.pdf)\n1. [Multi-Armed Bandits and the Stitch Fix Experimentation Platform](https:\u002F\u002Fmultithreaded.stitchfix.com\u002Fblog\u002F2020\u002F08\u002F05\u002Fbandits\u002F)\n1. [A\u002FB Testing Machine Learning Models](https:\u002F\u002Fmlinproduction.com\u002Fab-test-ml-models-deployment-series-08\u002F)\n1. [Data validation for machine learning. Polyzotis, N., Zinkevich, M., Roy, S., Breck, E. and Whang, S., 2019. Proceedings of Machine Learning and Systems](https:\u002F\u002Fmlsys.org\u002FConferences\u002F2019\u002Fdoc\u002F2019\u002F167.pdf)\n1. [Testing machine learning based systems: a systematic mapping](https:\u002F\u002Flink.springer.com\u002Fcontent\u002Fpdf\u002F10.1007\u002Fs10664-020-09881-0.pdf)\n1. [Explainable Monitoring: Stop flying blind and monitor your AI](https:\u002F\u002Fblog.fiddler.ai\u002F2020\u002F04\u002Fexplainable-monitoring-stop-flying-blind-and-monitor-your-ai\u002F)\n1. [WhyLogs: Embrace Data Logging Across Your ML Systems](https:\u002F\u002Fmedium.com\u002Fwhylabs\u002Fwhylogs-embrace-data-logging-a9449cd121d)\n1. [Evidently AI. Insights on doing machine learning in production. (Vendor blog.)](https:\u002F\u002Fevidentlyai.com\u002Fblog)\n1. [The definitive guide to comprehensively monitoring your AI](https:\u002F\u002Fwww.monalabs.io\u002Fmona-blog\u002Fdefinitiveguidetomonitorai)\n1. [Introduction to Unit Testing for Machine Learning](https:\u002F\u002Fthemlrebellion.com\u002Fblog\u002FIntroduction-To-Unit-Testing-Machine-Learning\u002F)\n1. [Production Machine Learning Monitoring: Outliers, Drift, Explainers & Statistical Performance](https:\u002F\u002Ftowardsdatascience.com\u002Fproduction-machine-learning-monitoring-outliers-drift-explainers-statistical-performance-d9b1d02ac158)\n1. Test-Driven Development in MLOps [Part 1](https:\u002F\u002Fmedium.com\u002Fmlops-community\u002Ftest-driven-development-in-mlops-part-1-8894575f4dec)\n1. [Domain-Specific Machine Learning Monitoring](https:\u002F\u002Fmedium.com\u002Fmlops-community\u002Fdomain-specific-machine-learning-monitoring-88bc0dd8a212)\n1. [Introducing ML Model Performance Management (Blog by fiddler)](https:\u002F\u002Fblog.fiddler.ai\u002F2021\u002F03\u002Fintroducing-ml-model-performance-management\u002F)\n1. [What is ML Observability? (Arize AI)](https:\u002F\u002Farize.com\u002Fwhat-is-ml-observability\u002F)\n1. [Beyond Monitoring: The Rise of Observability (Arize AI & Monte Carlo Data)](https:\u002F\u002Farize.com\u002Fbeyond-monitoring-the-rise-of-observability\u002F)\n1. [Model Failure Modes (Arize AI)](https:\u002F\u002Farize.com\u002Fml-model-failure-modes\u002F)\n1. [Quick Start to Data Quality Monitoring for ML (Arize AI)](https:\u002F\u002Farize.com\u002Fdata-quality-monitoring\u002F)\n1. [Playbook to Monitoring Model Performance in Production (Arize AI)](https:\u002F\u002Farize.com\u002Fmonitor-your-model-in-production\u002F)\n1. [Robust ML by Property Based Domain Coverage Testing (Blog by Efemarai)](https:\u002F\u002Ftowardsdatascience.com\u002Fwhy-dont-we-test-machine-learning-as-we-test-software-43f5720903d)\n1. [Monitoring and explainability of models in production](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2007.06299.pdf)\n1. [Beyond Monitoring: The Rise of Observability](https:\u002F\u002Faparnadhinak.medium.com\u002Fbeyond-monitoring-the-rise-of-observability-c53bdc1d2e0b)\n1. [ML Model Monitoring – 9 Tips From the Trenches. (by NU bank)](https:\u002F\u002Fbuilding.nubank.com.br\u002Fml-model-monitoring-9-tips-from-the-trenches\u002F)\n1. [Model health assurance at LinkedIn. By LinkedIn Engineering](https:\u002F\u002Fengineering.linkedin.com\u002Fblog\u002F2021\u002Fmodel-health-assurance-at-linkedin)\n1. [How to Trust Your Deep Learning Code](https:\u002F\u002Fkrokotsch.eu\u002Fcleancode\u002F2020\u002F08\u002F11\u002FUnit-Tests-for-Deep-Learning.html) ([Accompanying code](https:\u002F\u002Fgithub.com\u002Ftilman151\u002Funittest_dl))\n1. [Estimating Performance of Regression Models Without Ground-Truth](https:\u002F\u002Fbit.ly\u002Fmedium-estimating-performance-regression) (Using [NannyML](https:\u002F\u002Fbit.ly\u002Fml-ops-nannyml))\n1. [How Hyperparameter Tuning in Machine Learning Works (by NimbleBox.ai)](https:\u002F\u002Fnimblebox.ai\u002Fblog\u002Fhyperparameter-tuning-machine-learning)\n\u003C\u002Fdetails>\n\n\u003Ca name=\"mlops-infra\">\u003C\u002Fa>\n# MLOps: Infrastructure & Tooling\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [MLOps Infrastructure Stack Canvas](https:\u002F\u002Fmiro.com\u002Fapp\u002Fboard\u002Fo9J_lfoc4Hg=\u002F)\n1. [Rise of the Canonical Stack in Machine Learning. How a Dominant New Software Stack Will Unlock the Next Generation of Cutting Edge AI Apps](https:\u002F\u002Ftowardsdatascience.com\u002Frise-of-the-canonical-stack-in-machine-learning-724e7d2faa75)\n1. [AI Infrastructure Alliance. Building the canonical stack for AI\u002FML](https:\u002F\u002Fai-infrastructure.org\u002F)\n1. [Linux Foundation AI Foundation](https:\u002F\u002Fwiki.lfai.foundation\u002F)\n1. ML Infrastructure Tools for Production | [Part 1 — Production ML — The Final Stage of the Model Workflow](https:\u002F\u002Ftowardsdatascience.com\u002Fml-infrastructure-tools-for-production-1b1871eecafb) | [Part 2 — Model Deployment and Serving](https:\u002F\u002Ftowardsdatascience.com\u002Fml-infrastructure-tools-for-production-part-2-model-deployment-and-serving-fcfc75c4a362)\n1. [The MLOps Stack Template (by valohai)](https:\u002F\u002Fvalohai.com\u002Fblog\u002Fthe-mlops-stack\u002F)\n1. [Navigating the MLOps tooling landscape](https:\u002F\u002Fljvmiranda921.github.io\u002Fnotebook\u002F2021\u002F05\u002F10\u002Fnavigating-the-mlops-landscape\u002F)\n1. [MLOps.toys curated list of MLOps projects (by Aporia)](https:\u002F\u002Fmlops.toys\u002F)\n1. [Comparing Cloud MLOps platforms, From a former AWS SageMaker PM](https:\u002F\u002Ftowardsdatascience.com\u002Fcomparing-cloud-mlops-platform-from-a-former-aws-sagemaker-pm-115ced28239b)\n1. [Machine Learning Ecosystem 101 (whitepaper by Arize AI)](https:\u002F\u002Farize.com\u002Fwp-content\u002Fuploads\u002F2021\u002F04\u002FArize-AI-Ecosystem-White-Paper.pdf)\n1. [Selecting your optimal MLOps stack: advantages and challenges. By Intellerts](https:\u002F\u002Fintellerts.com\u002Fselecting-your-optimal-mlops-stack-advantages-and-challenges\u002F)\n1. [Infrastructure Design for Real-time Machine Learning Inference. The Databricks Blog](https:\u002F\u002Fdatabricks.com\u002Fblog\u002F2021\u002F09\u002F01\u002Finfrastructure-design-for-real-time-machine-learning-inference.html)\n1. [The 2021 State of AI Infrastructure Survey](https:\u002F\u002Fpages.run.ai\u002Fhubfs\u002FPDFs\u002F2021-State-of-AI-Infrastructure-Survey.pdf)\n1. [AI infrastructure Maturity matrix](https:\u002F\u002Fpages.run.ai\u002Fhubfs\u002FPDFs\u002FAI-Infrastructure-Maturity-Benchmarking-Model.pdf)\n1. [A Curated Collection of the Best Open-source MLOps Tools. By Censius](https:\u002F\u002Fcensius.ai\u002Fmlops-tools)\n1. [Best MLOps Tools to Manage the ML Lifecycle (by NimbleBox.ai)](https:\u002F\u002Fnimblebox.ai\u002Fblog\u002Fmlops-tools)\n1. [The minimum set of must-haves for MLOps](https:\u002F\u002Fmarvelousmlops.substack.com\u002Fp\u002Fthe-minimum-set-of-must-haves-for)\n\u003C\u002Fdetails>\n\n\n\u003Ca name=\"mlops-papers\">\u003C\u002Fa>\n# MLOps Papers\n\nA list of scientific and industrial papers and resources about Machine Learning operalization since 2015. [See more.](papers.md)\n\n\n\u003Ca name=\"talks-about-mlops\">\u003C\u002Fa>\n# Talks About MLOps\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [\"MLOps: Automated Machine Learning\" by Emmanuel Raj](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=m32k9jcY4pY)\n1. [DeliveryConf 2020. \"Continuous Delivery For Machine Learning: Patterns And Pains\" by Emily Gorcenski](https:\u002F\u002Fyoutu.be\u002FbFW5mZmj0nQ)\n1. [MLOps Conference: Talks from 2019](https:\u002F\u002Fwww.mlopsconf.com?wix-vod-comp-id=comp-k1ry4afh)\n1. [Kubecon 2019: Flyte: Cloud Native Machine Learning and Data Processing Platform](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=KdUJGSP1h9U)\n1. [Kubecon 2019: Running LargeScale Stateful workloads on Kubernetes at Lyft](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=ECeVQoble0g)\n1. [A CI\u002FCD Framework for Production Machine Learning at Massive Scale (using Jenkins X and Seldon Core)](https:\u002F\u002Fyoutu.be\u002F68_Phxwaj-k)\n1. [MLOps Virtual Event (Databricks)](https:\u002F\u002Fyoutu.be\u002F9Ehh7Vl7ByM)\n1. [MLOps NY conference 2019](https:\u002F\u002Fwww.iguazio.com\u002Fmlops-nyc-sessions\u002F)\n1. [MLOps.community YouTube Channel](https:\u002F\u002Fwww.youtube.com\u002Fchannel\u002FUCG6qpjVnBTTT8wLGBygANOQ)\n1. [MLinProduction YouTube Channel](https:\u002F\u002Fwww.youtube.com\u002Fchannel\u002FUC3B_Z9FTeu4i8xtxDjGaZxw)\n1. [Introducing MLflow for End-to-End Machine Learning on Databricks. Spark+AI Summit 2020. Sean Owen](https:\u002F\u002Fyoutu.be\u002Fnx3yFzx_nHI)\n1. [MLOps Tutorial #1: Intro to Continuous Integration for ML](https:\u002F\u002Fyoutu.be\u002F9BgIDqAzfuA)\n1. [Machine Learning At Speed: Operationalizing ML For Real-Time Data Streams (2019)](https:\u002F\u002Fyoutu.be\u002F46l_C7ibpuo)\n1. [Damian Brady - The emerging field of MLops](https:\u002F\u002Fhumansofai.podbean.com\u002Fe\u002Fdamian-brady-the-emerging-field-of-mlops\u002F)\n1. [MLOps - Entwurf, Entwicklung, Betrieb (INNOQ Podcast in German)](https:\u002F\u002Fwww.innoq.com\u002Fen\u002Fpodcast\u002F076-mlops\u002F)\n1. [Instrumentation, Observability & Monitoring of Machine Learning Models](https:\u002F\u002Fwww.infoq.com\u002Fpresentations\u002Finstrumentation-observability-monitoring-ml\u002F)\n1. [Efficient ML engineering: Tools and best practices](https:\u002F\u002Flearning.oreilly.com\u002Fvideos\u002Foreilly-strata-data\u002F9781492050681\u002F9781492050681-video327465?autoplay=false)\n1. [Beyond the jupyter notebook: how to build data science products](https:\u002F\u002Ftowardsdatascience.com\u002Fbeyond-the-jupyter-notebook-how-to-build-data-science-products-50d942fc25d8)\n1. [An introduction to MLOps on Google Cloud](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=6gdrwFMaEZ0#action=share) (First 19 min are vendor-, language-, and framework-agnostic. @visenger)\n1. [How ML Breaks: A Decade of Outages for One Large ML Pipeline](https:\u002F\u002Fyoutu.be\u002FhBMHohkRgAA)\n1. [Clean Machine Learning Code: Practical Software Engineering](https:\u002F\u002Fyoutu.be\u002FPEjTAJHxYPM)\n1. [Machine Learning Engineering: 10 Fundamentale Praktiken](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=VYlXNWxqJ2A)\n1. [Architecture of machine learning systems (3-part series)](https:\u002F\u002Fwww.youtube.com\u002Fplaylist?list=PLx8omXiw3n9y26FKZLV5ScyS52D_c29QN)\n1. [Machine Learning Design Patterns](https:\u002F\u002Fyoutu.be\u002FudXjlvCFusc)\n1. [The laylist that covers techniques and approaches for model deployment on to production](https:\u002F\u002Fyoutube.com\u002Fplaylist?list=PL3N9eeOlCrP5PlN1jwOB3jVZE6nYTVswk)\n1. [ML Observability: A Critical Piece in Ensuring Responsible AI (Arize AI at Re-Work)](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=2FE1sg749V[o)\n1. [ML Engineering vs. Data Science (Arize AI Un\u002FSummit)](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=lP_4lT2k7Kg&t=2s)\n1. [SRE for ML: The First 10 Years and the Next 10 ](https:\u002F\u002Fwww.usenix.org\u002Fconference\u002Fsrecon21\u002Fpresentation\u002Funderwood-sre-ml)\n1. [Demystifying Machine Learning in Production: Reasoning about a Large-Scale ML Platform](https:\u002F\u002Fwww.usenix.org\u002Fconference\u002Fsrecon21\u002Fpresentation\u002Fmcglohon)\n1. [Apply Conf 2022](https:\u002F\u002Fwww.applyconf.com\u002Fapply-conf-may-2022\u002F)\n1. [Databricks' Data + AI Summit 2022](https:\u002F\u002Fdatabricks.com\u002Fdataaisummit\u002Fnorth-america-2022)\n1. [RE•WORK MLOps Summit 2022](https:\u002F\u002Fwww.re-work.co\u002Fevents\u002Fmlops-summit-2022)\n1. [Annual MLOps World Conference](https:\u002F\u002Fmlopsworld.com\u002F)\n\u003C\u002Fdetails>\n\n\u003Ca name=\"existing-ml-systems\">\u003C\u002Fa>\n# Existing ML Systems\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [Introducing FBLearner Flow: Facebook’s AI backbone](https:\u002F\u002Fengineering.fb.com\u002Fml-applications\u002Fintroducing-fblearner-flow-facebook-s-ai-backbone\u002F)\n1. [TFX: A TensorFlow-Based Production-Scale Machine Learning Platform](https:\u002F\u002Fdl.acm.org\u002Fdoi\u002Fpdf\u002F10.1145\u002F3097983.3098021?download=true)\n1. [Accelerate your ML and Data workflows to production: Flyte](https:\u002F\u002Fflyte.org\u002F)\n1. [Getting started with Kubeflow Pipelines](https:\u002F\u002Fcloud.google.com\u002Fblog\u002Fproducts\u002Fai-machine-learning\u002Fgetting-started-kubeflow-pipelines)\n1. [Meet Michelangelo: Uber’s Machine Learning Platform](https:\u002F\u002Fwww.uber.com\u002Fblog\u002Fmichelangelo-machine-learning-platform\u002F)\n1. [Meson: Workflow Orchestration for Netflix Recommendations](https:\u002F\u002Fnetflixtechblog.com\u002Fmeson-workflow-orchestration-for-netflix-recommendations-fc932625c1d9)\n1. [What are Azure Machine Learning pipelines?](https:\u002F\u002Fdocs.microsoft.com\u002Fen-gb\u002Fazure\u002Fmachine-learning\u002Fconcept-ml-pipelines)\n1. [Uber ATG’s Machine Learning Infrastructure for Self-Driving Vehicles](https:\u002F\u002Feng.uber.com\u002Fmachine-learning-model-life-cycle-version-control\u002F)\n1. [An overview of ML development platforms](https:\u002F\u002Fwww.linkedin.com\u002Fpulse\u002Foverview-ml-development-platforms-louis-dorard\u002F)\n1. [Snorkel AI: Putting Data First in ML Development](https:\u002F\u002Fwww.snorkel.ai\u002F07-14-2020-snorkel-ai-launch.html)\n1. [A Tour of End-to-End Machine Learning Platforms](https:\u002F\u002Fdatabaseline.tech\u002Fa-tour-of-end-to-end-ml-platforms\u002F)\n1. [Introducing WhyLabs, a Leap Forward in AI Reliability](https:\u002F\u002Fmedium.com\u002Fwhylabs\u002Fintroducing-whylabs-5a3b4f37b998)\n1. [Project: Ease.ml (ETH Zürich)](https:\u002F\u002Fds3lab.inf.ethz.ch\u002Feaseml.html)\n1. [Bodywork: model-training and deployment automation](https:\u002F\u002Fbodywork.readthedocs.io\u002Fen\u002Flatest\u002F)\n1. [Lessons on ML Platforms — from Netflix, DoorDash, Spotify, and more](https:\u002F\u002Ftowardsdatascience.com\u002Flessons-on-ml-platforms-from-netflix-doordash-spotify-and-more-f455400115c7)\n1. [Papers & tech blogs by companies sharing their work on data science & machine learning in production. By Eugen Yan](https:\u002F\u002Fgithub.com\u002Feugeneyan\u002Fapplied-ml)\n1. [How do different tech companies approach building internal ML platforms? (tweet)](https:\u002F\u002Ftwitter.com\u002FEvidentlyAI\u002Fstatus\u002F1420328878585913344)\n1. [Declarative Machine Learning Systems](https:\u002F\u002Fdl.acm.org\u002Fdoi\u002Fpdf\u002F10.1145\u002F3475965.3479315)\n1. [StreamING Machine Learning Models: How ING Adds Fraud Detection Models at Runtime with Apache Flink](https:\u002F\u002Fwww.ververica.com\u002Fblog\u002Freal-time-fraud-detection-ing-bank-apache-flink)\n\u003C\u002Fdetails>\n\n\u003Ca name=\"machine-learning\">\u003C\u002Fa>\n# Machine Learning \n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. Book, Aurélien Géron,\"Hands-On Machine Learning with Scikit-Learn and TensorFlow\"\n1. [Foundations of Machine Learning](https:\u002F\u002Fbloomberg.github.io\u002Ffoml\u002F)\n1. [Best Resources to Learn Machine Learning](http:\u002F\u002Fwww.trainindatablog.com\u002Fbest-resources-to-learn-machine-learning\u002F)\n1. [Awesome TensorFlow](https:\u002F\u002Fgithub.com\u002Fjtoy\u002Fawesome-tensorflow)\n1. [\"Papers with Code\" - Browse the State-of-the-Art in Machine Learning](https:\u002F\u002Fpaperswithcode.com\u002Fsota)\n1. [Zhi-Hua Zhou. 2012. Ensemble Methods: Foundations and Algorithms. Chapman & Hall\u002FCRC.](https:\u002F\u002Fwww.amazon.com\u002Fexec\u002Fobidos\u002FASIN\u002F1439830037\u002Facmorg-20)\n1. [Feature Engineering for Machine Learning. Principles and Techniques for Data Scientists. By Alice Zheng, Amanda Casari](https:\u002F\u002Fwww.amazon.com\u002FFeature-Engineering-Machine-Learning-Principles-ebook\u002Fdp\u002FB07BNX4MWC)\n1. [Google Research: Looking Back at 2019, and Forward to 2020 and Beyond](https:\u002F\u002Fai.googleblog.com\u002F2020\u002F01\u002Fgoogle-research-looking-back-at-2019.html)\n1. [O’Reilly: The road to Software 2.0](https:\u002F\u002Fwww.oreilly.com\u002Fradar\u002Fthe-road-to-software-2-0\u002F)\n1. [Machine Learning and Data Science Applications in Industry](https:\u002F\u002Fgithub.com\u002Ffirmai\u002Findustry-machine-learning)\n1. [Deep Learning for Anomaly Detection](https:\u002F\u002Fff12.fastforwardlabs.com\u002F)\n1. [Federated Learning for Mobile Keyboard Prediction](https:\u002F\u002Farxiv.org\u002Fpdf\u002F1811.03604.pdf)\n1. [Federated Learning. Building better products with on-device data and privacy on default](https:\u002F\u002Ffederated.withgoogle.com\u002F)\n1. [Federated Learning: Collaborative Machine Learning without Centralized Training Data](https:\u002F\u002Fai.googleblog.com\u002F2017\u002F04\u002Ffederated-learning-collaborative.html) \n1. [Yang, Q., Liu, Y., Cheng, Y., Kang, Y., Chen, T. and Yu, H., 2019. Federated learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 13(3). Chapters 1 and 2.](https:\u002F\u002Fwww.morganclaypoolpublishers.com\u002Fcatalog_Orig\u002Fsamples\u002F9781681736983_sample.pdf)\n1. [Federated Learning by FastForward](https:\u002F\u002Ffederated.fastforwardlabs.com\u002F)\n1. [THE FEDERATED & DISTRIBUTED MACHINE LEARNING CONFERENCE](https:\u002F\u002Fwww.federatedlearningconference.com\u002F)\n1. [Federated Learning: Challenges, Methods, and Future Directions](https:\u002F\u002Fblog.ml.cmu.edu\u002F2019\u002F11\u002F12\u002Ffederated-learning-challenges-methods-and-future-directions\u002F)\n1. [Book: Molnar, Christoph. \"Interpretable machine learning. A Guide for Making Black Box Models Explainable\", 2019](https:\u002F\u002Fchristophm.github.io\u002Finterpretable-ml-book\u002F)\n1. [Book: Hutter, Frank, Lars Kotthoff, and Joaquin Vanschoren. \"Automated Machine Learning\". Springer,2019.](https:\u002F\u002Foriginalstatic.aminer.cn\u002Fmisc\u002Fpdf\u002FHutter-AutoML_Book_compressed.pdf)\n1. [ML resources by topic, curated by the community. ](https:\u002F\u002Fmadewithml.com\u002Ftopics\u002F)\n1. [An Introduction to Machine Learning Interpretability, by Patrick Hall, Navdeep Gill, 2nd Edition. O'Reilly 2019](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fan-introduction-to\u002F9781098115487\u002F)\n1. [Examples of techniques for training interpretable machine learning (ML) models, explaining ML models, and debugging ML models for accuracy, discrimination, and security.](https:\u002F\u002Fgithub.com\u002Fjphall663\u002Finterpretable_machine_learning_with_python)\n1. [Paper: \"Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence\", by Sebastian Raschka, Joshua Patterson, and Corey Nolet. 2020](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2002.04803.pdf)\n1. [Distill: Machine Learning Research](https:\u002F\u002Fdistill.pub\u002F)\n1. [AtHomeWithAI: Curated Resource List by DeepMind](https:\u002F\u002Fstorage.googleapis.com\u002Fdeepmind-media\u002Fresearch\u002FNew_AtHomeWithAI%20resources.pdf)\n1. [Awesome Data Science](https:\u002F\u002Fgithub.com\u002Facademic\u002Fawesome-datascience)\n1. [Intro to probabilistic programming. A use case using Tensorflow-Probability (TFP)](https:\u002F\u002Ftowardsdatascience.com\u002Fintro-to-probabilistic-programming-b47c4e926ec5)\n1. [Dive into Snorkel: Weak-Superversion on German Texts. inovex Blog](https:\u002F\u002Fwww.inovex.de\u002Fblog\u002Fsnorkel-weak-superversion-german-texts\u002F)\n1. [Dive into Deep Learning. An interactive deep learning book with code, math, and discussions. Provides NumPy\u002FMXNet, PyTorch, and TensorFlow implementations](http:\u002F\u002Fd2l.ai\u002F)\n1. [Data Science Collected Resources (GitHub repository)](https:\u002F\u002Fgithub.com\u002Ftirthajyoti\u002FData-science-best-resources)\n1. [Set of illustrated Machine Learning cheatsheets](https:\u002F\u002Fstanford.edu\u002F~shervine\u002Fteaching\u002Fcs-229\u002F)\n1. [\"Machine Learning Bookcamp\" by Alexey Grigorev](https:\u002F\u002Fwww.manning.com\u002Fbooks\u002Fmachine-learning-bookcamp)\n1. [130 Machine Learning Projects Solved and Explained](https:\u002F\u002Fmedium.com\u002Fthe-innovation\u002F130-machine-learning-projects-solved-and-explained-605d188fb392)\n1. [Machine learning cheat sheet](https:\u002F\u002Fgithub.com\u002Fsoulmachine\u002Fmachine-learning-cheat-sheet)\n1. [Stateoftheart AI. An open-data and free platform built by the research community to facilitate the collaborative development of AI](https:\u002F\u002Fwww.stateoftheart.ai\u002F)\n1. [Online Machine Learning Courses: 2020 Edition](https:\u002F\u002Fwww.blog.confetti.ai\u002Fpost\u002Fbest-online-machine-learning-courses-2020-edition)\n1. [End-to-End Machine Learning Library](https:\u002F\u002Fe2eml.school\u002Fblog.html)\n1. [Machine Learning Toolbox (by Amit Chaudhary)](https:\u002F\u002Famitness.com\u002Ftoolbox\u002F)\n1. [Causality for Machine Learning](https:\u002F\u002Fff13.fastforwardlabs.com\u002FFF13-Causality_for_Machine_Learning-Cloudera_Fast_Forward.pdf)\n1. [Causal Inference for the Brave and True](https:\u002F\u002Fmatheusfacure.github.io\u002Fpython-causality-handbook\u002Flanding-page.html)\n1. [Causal Inference](https:\u002F\u002Fmixtape.scunning.com\u002Findex.html)\n1. [A resource list for causality in statistics, data science and physics](https:\u002F\u002Fgithub.com\u002Fmsuzen\u002Flooper\u002Fblob\u002Fmaster\u002Flooper.md)\n1. [Learning from data. Caltech](http:\u002F\u002Fwork.caltech.edu\u002Flectures.html)\n1. [Machine Learning Glossary](https:\u002F\u002Fml-cheatsheet.readthedocs.io\u002Fen\u002Flatest\u002F#)\n1. [Book: \"Distributed Machine Learning Patterns\". 2022. By Yuan Tang. Manning](https:\u002F\u002Fwww.manning.com\u002Fbooks\u002Fdistributed-machine-learning-patterns)\n1. [Machine Learning for Beginners - A Curriculum](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FML-For-Beginners)\n1. [Making Friends with Machine Learning. By Cassie Kozyrkov]()\n1. [Machine Learning Workflow - A Complete Guide (by NimbleBox.ai)](https:\u002F\u002Fnimblebox.ai\u002Fblog\u002Fmachine-learning-workflow)\n1. [Performance Metrics to Monitor in Machine Learning Projects (by NimbleBox.ai)](https:\u002F\u002Fnimblebox.ai\u002Fblog\u002Fmachine-learning-performance-metrics)\n \n\u003C\u002Fdetails>\n\n\n\n\n\n\u003Ca name=\"software-engineering\">\u003C\u002Fa>\n# Software Engineering\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [The Twelve Factors](https:\u002F\u002F12factor.net\u002F)\n1. [Book \"Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations\", 2018 by Nicole Forsgren et.al](https:\u002F\u002Fwww.amazon.com\u002FAccelerate-Software-Performing-Technology-Organizations\u002Fdp\u002F1942788339)\n1. [Book \"The DevOps Handbook\" by Gene Kim, et al. 2016](https:\u002F\u002Fitrevolution.com\u002Fbook\u002Fthe-devops-handbook\u002F)\n1. [State of DevOps 2019](https:\u002F\u002Fresearch.google\u002Fpubs\u002Fpub48455\u002F)\n1. [Clean Code concepts adapted for machine learning and data science.](https:\u002F\u002Fgithub.com\u002Fdavified\u002Fclean-code-ml)\n1. [School of SRE](https:\u002F\u002Flinkedin.github.io\u002Fschool-of-sre\u002F)\n1. [10 Laws of Software Engineering That People Ignore](https:\u002F\u002Fwww.indiehackers.com\u002Fpost\u002F10-laws-of-software-engineering-that-people-ignore-e3439176dd)\n1. [The Patterns of Scalable, Reliable, and Performant Large-Scale Systems](http:\u002F\u002Fawesome-scalability.com\u002F)\n1. [The Book of Secret Knowledge](https:\u002F\u002Fgithub.com\u002Ftrimstray\u002Fthe-book-of-secret-knowledge)\n1. [SHADES OF CONWAY'S LAW](https:\u002F\u002Fthinkinglabs.io\u002Farticles\u002F2021\u002F05\u002F07\u002Fshades-of-conways-law.html)\n1. [Engineering Practices for Data Scientists](https:\u002F\u002Fvalohai.com\u002Fengineering-practices-ebook\u002F)\n\u003C\u002Fdetails>\n\n\n\u003Ca name=\"product-management-for-mlai\">\u003C\u002Fa>\n# Product Management for ML\u002FAI\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [What you need to know about product management for AI. A product manager for AI does everything a traditional PM does, and much more.](https:\u002F\u002Fwww.oreilly.com\u002Fradar\u002Fwhat-you-need-to-know-about-product-management-for-ai\u002F)\n1. [Bringing an AI Product to Market. Previous articles have gone through the basics of AI product management. Here we get to the meat: how do you bring a product to market?](https:\u002F\u002Fwww.oreilly.com\u002Fradar\u002Fbringing-an-ai-product-to-market\u002F)\n1. [The People + AI Guidebook](https:\u002F\u002Fpair.withgoogle.com\u002Fguidebook\u002F)\n1. [User Needs + Defining Success](https:\u002F\u002Fpair.withgoogle.com\u002Fchapter\u002Fuser-needs\u002F)\n1. [Building machine learning products: a problem well-defined is a problem half-solved.](https:\u002F\u002Fwww.jeremyjordan.me\u002Fml-requirements\u002F)\n1. [Talk: Designing Great ML Experiences (Apple)](https:\u002F\u002Fdeveloper.apple.com\u002Fvideos\u002Fplay\u002Fwwdc2019\u002F803\u002F) \n1. [Machine Learning for Product Managers](http:\u002F\u002Fnlathia.github.io\u002F2017\u002F03\u002FMachine-Learning-for-Product-Managers.html)\n1. [Understanding the Data Landscape and Strategic Play Through Wardley Mapping](https:\u002F\u002Fergestx.com\u002Fdata-landscape-wardley-mapping\u002F)\n1. [Techniques for prototyping machine learning systems across products and features](https:\u002F\u002Fdesign.google\u002Flibrary\u002Fsimulating-intelligence\u002F)\n1. [Machine Learning and User Experience: A Few Resources](https:\u002F\u002Fmedium.com\u002Fml-ux\u002Fmachine-learning-and-user-experience-a-few-resources-e7872f1d34ee)\n1. [AI ideation canvas](https:\u002F\u002Fidalab.de\u002Fwp-content\u002Fuploads\u002F2021\u002F02\u002Fidalab-AI-ideation-canvas-Feb21.pdf)\n1. [Ideation in AI](https:\u002F\u002Fidalab.de\u002Fideation-in-ai-five-ways-to-make-the-workshops-work\u002F)\n1. [5 Steps for Building Machine Learning Models for Business. By shopify engineering](https:\u002F\u002Fshopify.engineering\u002Fbuilding-business-machine-learning-models)\n1. [Metric Design for Data Scientists and Business Leaders](https:\u002F\u002Ftowardsdatascience.com\u002Fmetric-design-for-data-scientists-and-business-leaders-b8adaf46c00)\n\u003C\u002Fdetails>\n\n\n\u003Ca name=\"the-economics-of-mlai\">\u003C\u002Fa>\n# The Economics of ML\u002FAI\n\u003Cdetails>\n\u003Csummary>Click to expand!\u003C\u002Fsummary>\n \n1. [Book: \"Prediction Machines: The Simple Economics of Artificial Intelligence\"](https:\u002F\u002Fwww.predictionmachines.ai\u002F)\n1. [Book: \"The AI Organization\" by David Carmona](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fthe-ai-organization\u002F9781492057369\u002F)\n1. [Book: \"Succeeding with AI\". 2020. By Veljko Krunic. Manning Publications](https:\u002F\u002Flearning.oreilly.com\u002Flibrary\u002Fview\u002Fsucceeding-with-ai\u002F9781617296932\u002F)\n1. [A list of articles about AI and the economy](https:\u002F\u002Fwww.predictionmachines.ai\u002Farticles)\n1. [Gartner AI Trends 2019](https:\u002F\u002Fblogs.gartner.com\u002Fsmarterwithgartner\u002Ffiles\u002F2019\u002F08\u002FCTMKT_736691_Hype_Cycle_for_AI_2019.png)\n1. [Global AI Survey: AI proves its worth, but few scale impact](https:\u002F\u002Fwww.mckinsey.com\u002Ffeatured-insights\u002Fartificial-intelligence\u002Fglobal-ai-survey-ai-proves-its-worth-but-few-scale-impact)\n1. [Getting started with AI? Start here! Everything you need to know to dive into your project](https:\u002F\u002Fmedium.com\u002Fhackernoon\u002Fthe-decision-makers-guide-to-starting-ai-72ee0d7044df)\n1. [11 questions to ask before starting a successful Machine Learning project](https:\u002F\u002Ftryolabs.com\u002Fblog\u002F2019\u002F02\u002F13\u002F11-questions-to-ask-before-starting-a-successful-machine-learning-project\u002F)\n1. [What AI still can’t do](https:\u002F\u002Fwww.technologyreview.com\u002Fs\u002F615189\u002Fwhat-ai-still-cant-do\u002F)\n1. [Demystifying AI Part 4: What is an AI Canvas and how do you use it?](https:\u002F\u002Fwww.wearebrain.com\u002Fblog\u002Fai-data-science\u002Fwhat-is-an-ai-canvas\u002F)\n1. [A Data Science Workflow Canvas to Kickstart Your Projects](https:\u002F\u002Ftowardsdatascience.com\u002Fa-data-science-workflow-canvas-to-kickstart-your-projects-db62556be4d0)\n1. [Is your AI project a nonstarter? Here’s a reality check(list) to help you avoid the pain of learning the hard way](https:\u002F\u002Fmedium.com\u002Fhackernoon\u002Fai-reality-checklist-be34e2fdab9)\n1. [What is THE main reason most ML projects fail?](https:\u002F\u002Ftowardsdatascience.com\u002Fwhat-is-the-main-reason-most-ml-projects-fail-515d409a161f)\n1. [Designing great data products. The Drivetrain Approach: A four-step process for building data products.](https:\u002F\u002Fwww.oreilly.com\u002Fradar\u002Fdrivetrain-approach-data-products\u002F)\n1. [The New Business of AI (and How It’s Different From Traditional Software)](https:\u002F\u002Fa16z.com\u002F2020\u002F02\u002F16\u002Fthe-new-business-of-ai-and-how-its-different-from-traditional-software\u002F)\n1. [The idea maze for AI startups](https:\u002F\u002Fcdixon.org\u002F2015\u002F02\u002F01\u002Fthe-ai-startup-idea-maze)\n1. [The Enterprise AI Challenge: Common Misconceptions](https:\u002F\u002Fwww.forbes.com\u002Fsites\u002Fforbestechcouncil\u002F2020\u002F01\u002F15\u002Fthe-enterprise-ai-challenge-common-misconceptions\u002F#37ca1e5c5696)\n1. [Misconception 1 (of 5): Enterprise AI Is Primarily About The Technology](https:\u002F\u002Fwww.forbes.com\u002Fsites\u002Fforbestechcouncil\u002F2020\u002F01\u002F31\u002Fmisconception-1-of-5-enterprise-ai-is-primarily-about-the-technology\u002F#151e6711180e)\n1. [Misconception 2 (of 5): Automated Machine Learning Will Unlock Enterprise AI](https:\u002F\u002Fwww.forbes.com\u002Fsites\u002Ffor","awesome-mlops 是一个精心整理的MLOps（机器学习运维）资源列表。该项目汇集了从基础概念、书籍文章到工作流管理、数据工程、模型部署与服务等多方面的参考资料，涵盖了MLOps的核心技术和最佳实践。它适合于正在或将要从事机器学习项目开发和运维的专业人士，以及希望深入了解如何将机器学习模型高效地从设计阶段推进至生产环境并持续维护的团队。通过这个项目，用户可以快速找到关于MLOps各个方面的权威信息，从而加速自身在该领域的成长与发展。",2,"2026-06-11 03:23:53","top_topic"]