Data commingling, in computer science, occurs when different items or kinds of data are stored in such a way that they become commonly accessible when they are supposed to remain separated. In cloud computing, this can occur where different customer data sits on the same server. Data that is commingled can present a security vulnerability. Data commingling can also occur due to high speed data transmission mixing. In this situation, data of one security level can inadvertently or purposely be mixed with data of a lower or higher security level on the same transmission portal. Portal vehicles can be wire, fiber optics, microwave or various radio frequency transmission portals. This commingling can cause breaches of security and become a source of legal issues to any entity, corporation or individual. Data commingling can also occur when personal computers and personal software programs are used for business, security, government, etc. uses. In the early formulation stages of entities, non-profit or profit corporations, LLC's, LLP's, etc., the creation and use of stand-alone computers and stand-alone networks, "absolutely unconnected" to involved individuals, is the easiest, and safest way to prevent Data Commingling.
Legendre moment
In mathematics, Legendre moments are a type of image moment and are achieved by using the Legendre polynomial. Legendre moments are used in areas of image processing including: pattern and object recognition, image indexing, line fitting, feature extraction, edge detection, and texture analysis. Legendre moments have been studied as a means to reduce image moment calculation complexity by limiting the amount of information redundancy through approximation. == Legendre moments == Source: With order of m + n, and object intensity function f(x,y): L m n = ( 2 m + 1 ) ( 2 n + 1 ) 4 ∫ − 1 1 ∫ − 1 1 P m ( x ) P n ( y ) f ( x , y ) d x d y {\displaystyle L_{mn}={\frac {(2m+1)(2n+1)}{4}}\int \limits _{-1}^{1}\int \limits _{-1}^{1}P_{m}(x)P_{n}(y)f(x,y)\,dx\,dy} where m,n = 1, 2, 3, ...∞ with the nth-order Legendre polynomials being: P n ( x ) = ∑ k = 0 n a k , n x k = ( − 1 ) n 2 n n ! ( d d x ) [ ( 1 − x 2 ) n ] {\displaystyle P_{n}(x)=\sum _{k=0}^{n}a_{k,n}x^{k}={\frac {(-1)^{n}}{2^{n}n!}}\left({\frac {d}{dx}}\right)[(1-x^{2})^{n}]} which can also be written: P n ( x ) = ∑ k = 0 D ( n ) ( − 1 ) k ( 2 n − 2 k ) ! 2 n k ! ( n − k ) ! ( n − 2 k ) ! x n − 2 k = ( 2 n ) ! 2 n ( n ! ) 2 x n − ( 2 n − 2 ) ! 2 n 1 ! ( n − 1 ) ! ( n − 2 ) ! x n − 2 + ⋯ {\displaystyle {\begin{aligned}P_{n}(x)&=\sum _{k=0}^{D(n)}(-1)^{k}{\frac {(2n-2k)!}{2^{n}k!(n-k)!(n-2k)!}}x^{n-2k}\\[5pt]&={\frac {(2n)!}{2^{n}(n!)^{2}}}x^{n}-{\frac {(2n-2)!}{2^{n}1!(n-1)!(n-2)!}}x^{n-2}+\cdots \end{aligned}}} where D(n) = floor(n/2). The set of Legendre polynomials {Pn(x)} form an orthogonal set on the interval [−1,1]: ∫ − 1 1 P n ( x ) P m ( x ) d x = 2 2 n + 1 δ n m {\displaystyle \int _{-1}^{1}P_{n}(x)P_{m}(x)\,dx={\frac {2}{2n+1}}\delta _{nm}} A recurrence relation can be used to compute the Legendre polynomial: ( n + 1 ) P n + 1 ( x ) − ( 2 n + 1 ) x P n ( x ) + n P n − 1 ( x ) = 0 {\displaystyle (n+1)P_{n+1}(x)-(2n+1)xP_{n}(x)+nP_{n-1}(x)=0} f(x,y) can be written as an infinite series expansion in terms of Legendre polynomials [−1 ≤ x,y ≤ 1.]: f ( x , y ) = ∑ m = 0 ∞ ∑ n = 0 ∞ λ m n P m ( x ) P n ( y ) {\displaystyle f(x,y)=\sum _{m=0}^{\infty }\sum _{n=0}^{\infty }\lambda _{mn}P_{m}(x)P_{n}(y)}
Apache Parquet
Apache Parquet is a free and open-source column-oriented data storage format in the Apache Hadoop ecosystem inspired by Google Dremel interactive ad-hoc query system for analysis of read-only nested data. It is similar to RCFile and ORC, the other columnar-storage file formats in Hadoop, and is compatible with most of the data processing frameworks around Hadoop. It provides data compression and encoding schemes with enhanced performance to handle complex data in bulk. == History == The open-source project to build Apache Parquet began as a joint effort between Twitter and Cloudera using the record shredding and assembly algorithm as described in Google's Dremel. Parquet was designed as an improvement on the Trevni columnar storage format created by Doug Cutting, the creator of Hadoop. The name 'parquet' (lit. 'small compartment') refers to a style of decorative flooring and was chosen to "evoke the bottom layer of a database with an interesting layout". The first version, Apache Parquet 1.0, was released in July 2013. Since April 27, 2015, Apache Parquet has been a top-level Apache Software Foundation (ASF)-sponsored project. == Features == Apache Parquet is implemented using the record-shredding and assembly algorithm, which accommodates the complex data structures that can be used to store data. The values in each column are stored in contiguous memory locations, providing the following benefits: Column-wise compression is efficient in storage space Encoding and compression techniques specific to the type of data in each column can be used Queries that fetch specific column values need not read the entire row, thus improving performance Apache Parquet is implemented using the Apache Thrift framework, which increases its flexibility; it can work with a number of programming languages like C++, Java, Python, PHP, etc. As of August 2015, Parquet supports the big-data-processing frameworks including Apache Hive, Apache Drill, Apache Impala, Apache Crunch, Apache Pig, Cascading, Presto and Apache Spark. It is one of the external data formats used by the pandas Python data manipulation and analysis library. == Compression and encoding == In Parquet, compression is performed column by column, which enables different encoding schemes to be used for text and integer data. This strategy also keeps the door open for newer and better encoding schemes to be implemented as they are invented. Parquet supports various compression formats: snappy, gzip, LZO, brotli, zstd, and LZ4. === Dictionary encoding === Parquet has an automatic dictionary encoding enabled dynamically for data with a small number of unique values (i.e. below 105) that enables significant compression and boosts processing speed. === Bit packing === Storage of integers is usually done with dedicated 32 or 64 bits per integer. For small integers, packing multiple integers into the same space makes storage more efficient. === Run-length encoding (RLE) === To optimize storage of multiple occurrences of the same value, run-length encoding is used, which is where a single value is stored once along with the number of occurrences. Parquet implements a hybrid of bit packing and RLE, in which the encoding switches based on which produces the best compression results. This strategy works well for certain types of integer data and combines well with dictionary encoding. == Cloud Storage and Data Lakes == Parquet is widely used as the underlying file format in modern cloud-based data lake architectures. Cloud storage systems such as Amazon S3, Azure Data Lake Storage, and Google Cloud Storage commonly store data in Parquet format due to its efficient columnar representation and retrieval capabilities. Data lakehouse frameworks—including Apache Iceberg, Delta Lake, and Apache Hudi —build an additional metadata layer on top of Parquet files to support features such as schema evolution, time-travel queries, and ACID-compliant transactions. In these architectures, Parquet files serve as the immutable storage layer while the table formats manage data versioning and transactional integrity. == Comparison == Apache Parquet is comparable to RCFile and Optimized Row Columnar (ORC) file formats — all three fall under the category of columnar data storage within the Hadoop ecosystem. They all have better compression and encoding with improved read performance at the cost of slower writes. In addition to these features, Apache Parquet supports limited schema evolution, i.e., the schema can be modified according to the changes in the data. It also provides the ability to add new columns and merge schemas that do not conflict. Apache Arrow is designed as an in-memory complement to on-disk columnar formats like Parquet and ORC. The Arrow and Parquet projects include libraries that allow for reading and writing between the two formats. == Implementations == Known implementations of Parquet include:
Viaweb
Viaweb was a web-based application that allowed users to build and host their own online stores with little technical expertise using a web browser. The company was started in July 1995 by Paul Graham, Robert Morris (using the pseudonym "John McArtyem"), and Trevor Blackwell. Graham claims Viaweb was the first application service provider. Viaweb was also unusual for being partially written in the Lisp programming language. The software was originally called Webgen, but another company was using the same name, so the company renamed it to Viaweb, "because it worked via the Web". In 1998, Yahoo! Inc. bought Viaweb for 455,000 shares of Yahoo! capital stock, valued at about $49 million, and renamed it Yahoo! Store. Viaweb's example has been influential in Silicon Valley's entrepreneurial culture, largely due to Graham's widely read essays and his subsequent career as a successful venture capitalist.
Cloud Security Alliance
Cloud Security Alliance (CSA) is a not-for-profit organization with the mission to "promote the use of best practices for providing security assurance within cloud computing, artificial intelligence and to provide education on the uses of cloud computing to help secure all other forms of computing." The CSA has over 80,000 individual members worldwide. The CSA gained significant reputability in 2011 when the American Presidential Administration selected the CSA Summit as the venue for announcing the federal government’s cloud computing strategy. == History == The CSA was formed in December 2008 as a coalition by individuals who saw the need to provide objective enterprise user guidance on the adoption and use of cloud computing. Its initial work product, Security Guidance for Critical Areas of Focus in Cloud Computing, was put together in a Wiki-style by dozens of volunteers. In 2014, the Chairman of the Board of the CSA was Dave Cullinane, VP of Global Security and Privacy for Catalina Marketing, St. Petersburg, Florida, and former CISO for eBay. Cullinane has said, "If you have an application exposed to the Internet that will allow people to make money, it will be probed." == Profile == In 2009, the Cloud Security Alliance incorporated in Nevada as a Corporation and achieved US Federal 501(c)6 non-profit status. It is registered as a Foreign Non-Profit Corporation in Washington. == Policy maker support == The CSA works to support a number of global policy makers in their focus on cloud security initiatives including the National Institute of Standards and Technology (NIST), European Commission, Singapore Government, and other data protection authorities. In March 2012, the CSA was selected to partner with three of Europe’s largest research centers (CERN, EMBL and ESA) to launch Helix Nebula – The Science Cloud. == Size == The Cloud Security Alliance employs roughly sixty full-time and contract staff worldwide. It has several thousand active volunteers participating in research, working groups and chapters at any time. == Membership == According to CSA, they are a member-driven organization, chartered with promoting the use of best practices for providing security assurance within Cloud Computing, and providing education on the uses of Cloud Computing to help secure all other forms of computing. === Individuals === Individuals who are interested in cloud computing and have experience to assist in making it more secure receive a complimentary individual membership based on a minimum level of participation. === Chapters === The Cloud Security Alliance has a network of chapters worldwide. Chapters are separate legal entities from the Cloud Security Alliance, but operate within guidelines set down by the Cloud Security Alliance In the United States, Chapters may elect to benefit from the non-profit tax shield that the Cloud Security Alliance has. Chapters are encouraged to hold local meetings and participate in areas of research. Chapter activities are coordinated by the Cloud Security Alliance worldwide. === International scope === There are separate legal entities in Europe and Asia Pacific, called Cloud Security Alliance (Europe), a Scottish company in the United Kingdom, and Cloud Security Alliance Asia Pacific Ltd, in Singapore. Each legal entity is responsible for overseeing all Cloud Security Alliance-related activities in their respective regions. These legal entities operate under an agreement with Cloud Security Alliance that give it oversight power and have separate Boards of Directors. Both are companies Limited By Guarantee. The Managing Directors of each are members of the Executive Team of Cloud Security Alliance. == Areas of research == The Cloud Security Alliance has 25+ active working groups. Key areas of research include cloud standards, certification, education and training, guidance and tools, global reach, and driving innovation. Security Guidance for Critical Areas of Focus in Cloud Computing. Foundational best practices for securing cloud computing. Top Threats to Cloud Computing. Helps organizations make educated risk management decisions regarding their cloud adoption strategies. GRC (Governance, Risk and Compliance) Stack. A toolkit for key stakeholders to instrument and assess clouds against industry established best practices, standards and critical compliance requirements. Cloud Controls Matrix (CCM). Security controls framework for cloud provider and cloud consumers. CloudTrust Protocol. The mechanism by which cloud service consumers ask for and receive information about the elements of transparency as applied to cloud service providers. Consensus Assessments Initiative Research. Tools and processes to perform consistent measurements of cloud providers. Software Defined Perimeter. A proposed security framework that can be deployed to protect application infrastructure from network-based attacks. It will incorporate standards from organizations such as OASIS and NIST and security concepts from organizations like the U.S. DoD into an integrated framework. == Working groups and initiatives == Mobile Working Group Big Data Working Group Security as a Service Working Group Trusted Cloud Initiative CloudAudit CloudCERT CloudSIRT Cloud Metrics Security, Trust and Assurance Registry (STAR) Cloud Data Governance Turbot (business) Blockchain/Distributed Ledger
MLOps
MLOps or ML Ops is a paradigm that aims to deploy and maintain machine learning models in production reliably and efficiently. It bridges the gap between machine learning development and production operations, ensuring that models are robust, scalable, and aligned with business goals. The word is a compound of "machine learning" and the continuous delivery practice (CI/CD) of DevOps in the software field. Machine learning models are tested and developed in isolated experimental systems. When an algorithm is ready to be launched, MLOps is practiced between data scientists, DevOps, and machine learning engineers to transition the algorithm to production systems. Similar to DevOps or DataOps approaches, MLOps seeks to increase automation and improve the quality of production models, while also focusing on business and regulatory requirements. While MLOps started as a set of best practices, it is slowly evolving into an independent approach to ML lifecycle management. MLOps applies to the entire lifecycle - from integrating with model generation (software development lifecycle, continuous integration/continuous delivery), orchestration, and deployment, to health, diagnostics, governance, and business metrics. == Definition == MLOps is a paradigm, including aspects like best practices, sets of concepts, as well as a development culture when it comes to the end-to-end conceptualization, implementation, monitoring, deployment, and scalability of machine learning products. Most of all, it is an engineering practice that leverages three contributing disciplines: machine learning, software engineering (especially DevOps), and data engineering. MLOps is aimed at productionizing machine learning systems by bridging the gap between development (Dev) and operations (Ops). Essentially, MLOps aims to facilitate the creation of machine learning products by leveraging these principles: CI/CD automation, workflow orchestration, reproducibility; versioning of data, model, and code; collaboration; continuous ML training and evaluation; ML metadata tracking and logging; continuous monitoring; and feedback loops. == History == Interest in operationalizing machine learning systems began to grow in the mid-2010s as ML projects started moving from experimentation to production use. The challenges associated with sustaining such systems were highlighted in a 2015 paper. The predicted growth in machine learning included an estimated doubling of ML pilots and implementations from 2017 to 2018, and again from 2018 to 2020. Reports show a majority (up to 88%) of corporate machine learning initiatives are struggling to move beyond test stages. However, those organizations that actually put machine learning into production saw a 3–15% profit margin increases. The MLOps market size was USD 2,191.8 Million in 2024, and is projected to be USD 16,613.4 Million in 2030. == Architecture == Machine Learning systems can be categorized in eight different categories: data collection, data processing, feature engineering, data labeling, model design, model training and optimization, endpoint deployment, and endpoint monitoring. Each step in the machine learning lifecycle is built in its own system, but requires interconnection. These are the minimum systems that enterprises need to scale machine learning within their organization. == Goals == There are a number of goals enterprises want to achieve through MLOps systems successfully implementing ML across the enterprise, including: Deployment and automation Reproducibility of models and predictions Diagnostics Governance and regulatory compliance Scalability Collaboration Business uses Monitoring and management A standard practice, such as MLOps, takes into account each of the aforementioned areas, which can help enterprises optimize workflows and avoid issues during implementation. Vendors such as Adaptive ML deliver commercial reinforcement learning operations (RLOps) and MLOps-infrastructure, targeting organizations deploying large language models in production. A common architecture of an MLOps system would include data science platforms where models are constructed and the analytical engines where computations are performed, with the MLOps tool orchestrating the movement of machine learning models, data and outcomes between the systems.
Managed private cloud
Managed private cloud (also known as "hosted private cloud" or "single-tenant SaaS") refers to a principle in software architecture where a single instance of the software runs on a server, serves a single client organization (tenant), and is managed by a third party. The third-party provider is responsible for providing the hardware for the server and also for preliminary maintenance. This is in contrast to multitenancy, where multiple client organizations share a single server, or an on-premises deployment, where the client organization hosts its software instance. Managed private clouds also fall under the larger umbrella of cloud computing. == Adoption == The need for private clouds arose due to enterprises requiring a dedicated service and infrastructure for their cloud computing needs, such as for business-critical operations, improved security, and better control over their resources. Managed private cloud adoption is a popular choice among organizations. It has been on the rise due to enterprises requiring a dedicated cloud environment and preferring to avoid having to deal with management, maintenance, or future upgrade costs for the associated infrastructure and services. Such operational costs are unavoidable in on-premises private cloud data centers. == Advantages and challenges of managed private cloud == A managed private cloud cuts down on upkeep costs by outsourcing infrastructure management and maintenance to the managed cloud provider. It is easier to integrate an organization's existing software, services, and applications into a dedicated cloud hosting infrastructure which can be customized to the client's needs instead of a public cloud platform, whose hardware or infrastructure/software platform cannot be individualized to each client. Customers who choose a managed private cloud deployment usually choose them because of their desire for efficient cloud deployment, but also have the need for service customization or integration only available in a single-tenant environment. This chart shows the key benefits of the different types of deployments, and shows the overlap between these cloud solutions. This chart shows key drawbacks. Since deployments are done in a single-tenant environment, it is usually cost-prohibitive for small and medium-sized businesses. While server upkeep and maintenance are handled by the service provider, including network management and security, the client is charged for all such services. It is up to the potential client to determine if a managed private cloud solution aligns with their business objectives and budget. While the service provider maintains the upkeep of servers, network, and platform infrastructure, sensitive data is typically not stored on managed private clouds as it may leave business-critical information prone to breaches via third-party attacks on the cloud service provider. Common customizations and integrations include: Active Directory Single Sign-on Learning Management Systems Video Teleconferencing == Deployment strategies and service providers == Software companies have taken a variety of strategies in the Managed Private Cloud realm. Some software organizations have provided managed private cloud options internally, such as Microsoft. Companies that offer an on-premises deployment option, by definition, enable third-party companies to market Managed Private Cloud solutions. A few managed private cloud service providers are: Adobe Connect: Adobe Connect may be purchased for on-premises deployment, multi-tenant hosted deployment, managed private cloud as ACMS, or managed by third-party managed private cloud provider ConnectSolutions. Rackspace CenturyLink Microsoft licenses for Lync, SharePoint and Exchange may be purchased for on-premises deployment, a multi-tenant hosted deployment via Office 365, or managed by third-party cloud hosting from Azaleos, ConnectSolutions and others.