fbpx
Wikipedia

Software repository

This article contains embedded lists that may be poorly defined, unverified or indiscriminate. Please help to clean it up to meet Wikipedia's quality standards. Where appropriate, incorporate items into the main body of the article.(March 2019)

A software repository, or “repo” for short, is a storage location for software packages. Often a table of contents is also stored, along with metadata. A software repository is typically managed by source control or repository managers. Package Managers allow for installing and updating the repositories (sometimes called “packages”) versus having to do this manually.

Contents

Many software publishers and other organizations maintain servers on the Internet for this purpose, either free of charge or for a subscription fee. Repositories may be solely for particular programs, such as CPAN for the Perl programming language, or for an entire operating system. Operators of such repositories typically provide a package management system, tools intended to search for, install and otherwise manipulate software packages from the repositories. For example, many Linux distributions use Advanced Packaging Tool (APT), commonly found in Debian based distributions, or yum found in Red Hat based distributions. There are also multiple independent package management systems, such as pacman, used in Arch Linux and equo, found in Sabayon Linux.

As software repositories are designed to include useful packages, major repositories are designed to be malware free. If a computer is configured to use a digitally signed repository from a reputable vendor, and is coupled with an appropriate permissions system, this significantly reduces the threat of malware to these systems. As a side effect, many systems that have these capabilities do not require anti-malware software such as anti-virus software.

Most major Linux distributions have many repositories around the world that mirror the main repository.

In an enterprise environment, a software repository is usually used to store artifacts, or to mirror external repositories which may be inaccessible due to security restrictions. Such repositories may provide additional functionality, like access control, versioning, security checks for uploaded software, cluster functionality etc. and typically support a variety of formats in one package, so as to cater for all the needs in an enterprise, and thus aiming to provide a single point of truth. Popular examples are JFrog Artifactory and Nexus repository.

At client side, a package manager helps installing from and updating the repositories.

At server side, a software repository is typically managed by source control or repository managers. Some of the repository managers allow to aggregate other repository location into one URL and provide a caching proxy. When doing continuous builds many artifacts are produced and often centrally stored, so automatically deleting the ones which are not released is important.

A package management system is different from a package development process.

A typical use of a package management system is to facilitate the integration of code from possibly different sources into a coherent stand-alone operating unit. Thus, a package management system might be used to produce a distribution of Linux, possibly a distribution tailored to a specific restricted application.

A package development process, by contrast, is used to manage the co-development of code and documentation of a collection of functions or routines with a common theme, producing thereby a package of software functions that typically will not be complete and usable by themselves. A good package development process will help users conform to good documentation and coding practices, integrating some level of unit testing.

The following table lists a few languages with repositories for contributed software. The "Autochecks" column describes the routine checks done.

Very few people have the ability to test their software under multiple operating systems with different versions of the core code and with other contributed packages they may use. For the R programming language, the Comprehensive R Archive Network (CRAN) runs tests routinely.

To understand how this is valuable, imagine a situation with two developers, Sally and John. Sally contributes a package A. Sally only runs the current version of the software under one version of Microsoft Windows, and has only tested it in that environment. At more or less regular intervals, CRAN tests Sally's contribution under a dozen combinations of operating systems and versions of the core R language software. If one of them generates an error, she gets that error message. With luck, that error message details may provide enough input to allow enable a fix for the error, even if she cannot replicate it with her current hardware and software. Next, suppose John contributes to the repository a package B that uses a package A. Package B passes all the tests and is made available to users. Later, Sally submits an improved version of A, which unfortunately, breaks B. The autochecks make it possible to provide information to John so he can fix the problem.

This example exposes both a strength and a weakness in the R contributed-package system: CRAN supports this kind of automated testing of contributed packages, but packages contributed to CRAN need not specify the versions of other contributed packages that they use. Procedures for requesting specific versions of packages exist, but contributors might not use those procedures.

Beyond this, a repository such as CRAN running regular checks of contributed packages actually provides an extensive if ad hoc test suite for development versions of the core language. If Sally (in the example above) gets an error message she does not understand or thinks is inappropriate, especially from a development version of the language, she can (and often does with R) ask the core development-team for the language for help. In this way, the repository can contribute to improving the quality of the core language software.

Language / purpose Package Development Process Repository Install methods Collaborative development platform Autochecks
Haskell Common Architecture for Building Applications and Libraries Hackage cabal (software)
Java Maven
Julia
Common Lisp Quicklisp
.NET NuGet NuGet
Node.js npm
Perl CPAN PPM
PHP PEAR, Composer PECL, Packagist
Python Setuptools PyPI pip, EasyInstall, PyPM, Anaconda
R R CMD check process CRAN install.packages
remotes
GitHub
Frequently on 12 platforms or combinations of different versions of R (devel, prerel, patched, release) on different operating systems (different versions of Linux, Windows, macOS, and Solaris).
Ruby RubyGems Ruby Application Archive RubyForge
Rust Cargo Crates Cargo
TeX, LaTeX CTAN

(Parts of this table were copied from a "List of Top Repositories by Programming Language" on Stack Overflow)

Many other programming languages, among them C, C++, and Fortran, do not possess a central software repository with universal scope. Notable repositories with limited scope include:

  • Netlib, mainly mathematical routines for Fortran and C, historically one of the first open software repositories;
  • Boost, a strictly curated collection of high-quality libraries for C++; some code developed in Boost later became part of the C++ standard library.
Main article: Package manager

Package managers help manage repositories and the distribution of them. If a repository is updated, a package manager will typically allow the user to update that repository through the package manager. They also help with managing things such as dependencies between other software repositories. Some examples of Package Managers include:

Popular Package Managers
Package Manager Description
npm A package manager for Node.js
pip A package installer for Python
apt For managing Debian Packages
Homebrew A package installer for MacOS that allows you to install packages Apple didn't
vcpkg A package manager for C and C++
yum and dnf Package manager for Fedora and Red Hat Enterprise Linux
pacman Package manager for Arch Linux

Relationship to continuous integration

As part of the development lifecycle, source code is continuously being built into binary artifacts using continuous integration. This may interact with a binary repository manager much like a developer would by getting artifacts from the repositories and pushing builds there. Tight integration with CI servers enables the storage of important metadata such as:

  • Which user triggered the build (whether manually or by committing to revision control)
  • Which modules were built
  • Which sources were used (commit id, revision, branch)
  • Dependencies used
  • Environment variables
  • Packages installed

Artifacts and packages

Artifacts and packages inherently mean different things. Artifacts are simply an output or collection of files (ex. JAR, WAR, DLLS, RPM etc.) and one of those files may contain metadata (e.g. POM file). Whereas packages are a single archive file in a well-defined format (ex. NuGet) that contain files appropriate for the package type (ex. DLL, PDB). Many artifacts result from builds but other types are crucial as well. Packages are essentially one of two things: a library or an application.

Compared to source files, binary artifacts are often larger by orders of magnitude, they are rarely deleted or overwritten (except for rare cases such as snapshots or nightly builds), and they are usually accompanied by much metadata such as id, package name, version, license and more.

Metadata

Metadata describes a binary artifact, is stored and specified separately from the artifact itself, and can have several additional uses. The following table shows some common metadata types and their uses:

Metadata type Used for
Versions available Upgrading and downgrading automatically
Dependencies Specify other artifacts that the current artifact depends on
Downstream dependencies Specify other artifacts that depend on the current artifact
License Legal compliance
Build date and time Traceability
Documentation Provide offline availability for contextual documentation in IDEs
Approval information Traceability
Metrics Code coverage, compliance to rules, test results
User-created metadata Custom reports and processes
  1. itmWEB: Coping with Computer Viruses Archived October 14, 2007, at the Wayback Machine
  2. "JFrog Artifactory - wikieduonline".
  3. "Artifactory - Universal Artifact Management".
  4. "Nexus Repository | Software Component Management".
  5. "The Haskell Cabal | Overview". www.haskell.org. Retrieved2019-03-25.
  6. "Maven – Welcome to Apache Maven". maven.apache.org. Retrieved2019-03-25.
  7. "Julia Package Listing". pkg.julialang.org. Retrieved2019-03-25.
  8. "Quicklisp beta". www.quicklisp.org. Retrieved2019-03-25.
  9. karann-msft. "NuGet Package Manager UI Reference". docs.microsoft.com. Retrieved2019-03-25.
  10. "npm". www.npmjs.com. Retrieved2019-03-25.
  11. "Installing Perl Modules - www.cpan.org". www.cpan.org. Retrieved2019-03-25.
  12. Leisch, Friedrich. "Creating R Packages: A Tutorial"(PDF).
  13. Graves, Spencer B.; Dorai-Raj, Sundar. "Creating R Packages, Using CRAN, R-Forge, And Local R Archive Networks And Subversion (SVN) Repositories"(PDF).
  14. "The Comprehensive R Archive Network". cran.r-project.org. Retrieved2019-03-25.
  15. "R Installation and Administration". cran.r-project.org. Retrieved2019-03-25.
  16. Wickham, Hadley; Bryan, Jenny. "Package structure and state". R Packages. O'Reilly.
  17. Decan, Alexandre; Mens, Tom; Claes, Maelick; Grosjean, Philippe (2015). "On the Development and Distribution of R Packages: An Empirical Analysis of the R Ecosystem". Proceedings of the 2015 European Conference on Software Architecture Workshops - ECSAW '15: 1–6. doi:10.1145/2797433.2797476. S2CID 1680582.
  18. "The Cargo Book". Documentation. Rust Programming Language. Retrieved2019-08-26.
  19. "Rust Package Registry". crates.io. Retrieved2019-08-26.
  20. "List of Top Repositories by Programming Language". Stack Overflow. Retrieved2010-04-14.
  21. "npm About". www.npmjs.com. Retrieved2019-11-21.
  22. developers, The pip, pip: The PyPA recommended tool for installing Python packages., retrieved2019-11-21
  23. "Apt - Debian Wiki". wiki.debian.org. Retrieved2019-11-22.
  24. "Homebrew". Homebrew. Retrieved2019-11-22.
  25. "Yelp launches Yelp Fusion, Microsoft creates Vcpkg tool, and the new Touch Sense SDK for Android developers". SD Times. September 20, 2016.
  26. "Microsoft's C++ library manager now available for Linux and macOS". SD Times. April 25, 2018.
  27. Chinthaguntla, Keerthi. "Linux package management with YUM and RPM". Enable Sysadmin. Retrieved2021-04-11.
  28. "pacman - ArchWiki". wiki.archlinux.org. Retrieved2021-04-11.
  29. Chris, Tucker (2007-03-15). "Optimal Package Install/Uninstall Manager"(PDF). UC San Diego: 1. Retrieved2011-09-14.Cite journal requires |journal= ()
  30. "Linux repository classification schemes". braintickle.blogspot.com. 13 January 2006. Retrieved2008-03-01.

Software repository
Software repository Language Watch Edit This article contains embedded lists that may be poorly defined unverified or indiscriminate Please help to clean it up to meet Wikipedia s quality standards Where appropriate incorporate items into the main body of the article March 2019 A software repository or repo for short is a storage location for software packages Often a table of contents is also stored along with metadata A software repository is typically managed by source control or repository managers Package Managers allow for installing and updating the repositories sometimes called packages versus having to do this manually Contents 1 Overview 2 Package management system vs package development process 3 Selected repositories 4 Package managers 5 Repository managers 5 1 Relationship to continuous integration 5 1 1 Artifacts and packages 5 1 2 Metadata 6 See also 7 ReferencesOverview EditMany software publishers and other organizations maintain servers on the Internet for this purpose either free of charge or for a subscription fee Repositories may be solely for particular programs such as CPAN for the Perl programming language or for an entire operating system Operators of such repositories typically provide a package management system tools intended to search for install and otherwise manipulate software packages from the repositories For example many Linux distributions use Advanced Packaging Tool APT commonly found in Debian based distributions or yum found in Red Hat based distributions There are also multiple independent package management systems such as pacman used in Arch Linux and equo found in Sabayon Linux As software repositories are designed to include useful packages major repositories are designed to be malware free If a computer is configured to use a digitally signed repository from a reputable vendor and is coupled with an appropriate permissions system this significantly reduces the threat of malware to these systems As a side effect many systems that have these capabilities do not require anti malware software such as anti virus software 1 Most major Linux distributions have many repositories around the world that mirror the main repository In an enterprise environment a software repository is usually used to store artifacts or to mirror external repositories which may be inaccessible due to security restrictions Such repositories may provide additional functionality like access control versioning security checks for uploaded software cluster functionality etc and typically support a variety of formats in one package so as to cater for all the needs in an enterprise and thus aiming to provide a single point of truth Popular examples are JFrog Artifactory 2 3 and Nexus repository 4 At client side a package manager helps installing from and updating the repositories At server side a software repository is typically managed by source control or repository managers Some of the repository managers allow to aggregate other repository location into one URL and provide a caching proxy When doing continuous builds many artifacts are produced and often centrally stored so automatically deleting the ones which are not released is important Package management system vs package development process EditA package management system is different from a package development process A typical use of a package management system is to facilitate the integration of code from possibly different sources into a coherent stand alone operating unit Thus a package management system might be used to produce a distribution of Linux possibly a distribution tailored to a specific restricted application A package development process by contrast is used to manage the co development of code and documentation of a collection of functions or routines with a common theme producing thereby a package of software functions that typically will not be complete and usable by themselves A good package development process will help users conform to good documentation and coding practices integrating some level of unit testing Selected repositories EditThe following table lists a few languages with repositories for contributed software The Autochecks column describes the routine checks done Very few people have the ability to test their software under multiple operating systems with different versions of the core code and with other contributed packages they may use For the R programming language the Comprehensive R Archive Network CRAN runs tests routinely To understand how this is valuable imagine a situation with two developers Sally and John Sally contributes a package A Sally only runs the current version of the software under one version of Microsoft Windows and has only tested it in that environment At more or less regular intervals CRAN tests Sally s contribution under a dozen combinations of operating systems and versions of the core R language software If one of them generates an error she gets that error message With luck that error message details may provide enough input to allow enable a fix for the error even if she cannot replicate it with her current hardware and software Next suppose John contributes to the repository a package B that uses a package A Package B passes all the tests and is made available to users Later Sally submits an improved version of A which unfortunately breaks B The autochecks make it possible to provide information to John so he can fix the problem This example exposes both a strength and a weakness in the R contributed package system CRAN supports this kind of automated testing of contributed packages but packages contributed to CRAN need not specify the versions of other contributed packages that they use Procedures for requesting specific versions of packages exist but contributors might not use those procedures Beyond this a repository such as CRAN running regular checks of contributed packages actually provides an extensive if ad hoc test suite for development versions of the core language If Sally in the example above gets an error message she does not understand or thinks is inappropriate especially from a development version of the language she can and often does with R ask the core development team for the language for help In this way the repository can contribute to improving the quality of the core language software Language purpose Package Development Process Repository Install methods Collaborative development platform AutochecksHaskell Common Architecture for Building Applications and Libraries 5 Hackage cabal software Java Maven 6 Julia 7 Common Lisp Quicklisp 8 NET NuGet NuGet 9 Node js npm 10 Perl CPAN PPM 11 PHP PEAR Composer PECL PackagistPython Setuptools PyPI pip EasyInstall PyPM AnacondaR R CMD check process 12 13 CRAN 14 install packages 15 remotes 16 GitHub 17 Frequently on 12 platforms or combinations of different versions of R devel prerel patched release on different operating systems different versions of Linux Windows macOS and Solaris Ruby RubyGems Ruby Application Archive RubyForgeRust Cargo 18 Crates 19 Cargo 18 TeX LaTeX CTAN Parts of this table were copied from a List of Top Repositories by Programming Language on Stack Overflow 20 Many other programming languages among them C C and Fortran do not possess a central software repository with universal scope Notable repositories with limited scope include Netlib mainly mathematical routines for Fortran and C historically one of the first open software repositories Boost a strictly curated collection of high quality libraries for C some code developed in Boost later became part of the C standard library Package managers EditMain article Package manager Package managers help manage repositories and the distribution of them If a repository is updated a package manager will typically allow the user to update that repository through the package manager They also help with managing things such as dependencies between other software repositories Some examples of Package Managers include Popular Package Managers Package Manager Descriptionnpm A package manager for Node js 21 pip A package installer for Python 22 apt For managing Debian Packages 23 Homebrew A package installer for MacOS that allows you to install packages Apple didn t 24 vcpkg A package manager for C and C 25 26 yum and dnf Package manager for Fedora and Red Hat Enterprise Linux 27 pacman Package manager for Arch Linux 28 Repository managers EditRelationship to continuous integration Edit As part of the development lifecycle source code is continuously being built into binary artifacts using continuous integration This may interact with a binary repository manager much like a developer would by getting artifacts from the repositories and pushing builds there Tight integration with CI servers enables the storage of important metadata such as Which user triggered the build whether manually or by committing to revision control Which modules were built Which sources were used commit id revision branch Dependencies used Environment variables Packages installedArtifacts and packages Edit Artifacts and packages inherently mean different things Artifacts are simply an output or collection of files ex JAR WAR DLLS RPM etc and one of those files may contain metadata e g POM file Whereas packages are a single archive file in a well defined format ex NuGet that contain files appropriate for the package type ex DLL PDB 29 Many artifacts result from builds but other types are crucial as well Packages are essentially one of two things a library or an application 30 Compared to source files binary artifacts are often larger by orders of magnitude they are rarely deleted or overwritten except for rare cases such as snapshots or nightly builds and they are usually accompanied by much metadata such as id package name version license and more Metadata Edit Metadata describes a binary artifact is stored and specified separately from the artifact itself and can have several additional uses The following table shows some common metadata types and their uses Metadata type Used forVersions available Upgrading and downgrading automaticallyDependencies Specify other artifacts that the current artifact depends onDownstream dependencies Specify other artifacts that depend on the current artifactLicense Legal complianceBuild date and time TraceabilityDocumentation Provide offline availability for contextual documentation in IDEsApproval information TraceabilityMetrics Code coverage compliance to rules test resultsUser created metadata Custom reports and processesSee also EditPackage manager RPM Package Manager Synaptic FreeBSD Ports Definitive Media Library dpkg Simtel APTonCDReferences Edit itmWEB Coping with Computer Viruses Archived October 14 2007 at the Wayback Machine JFrog Artifactory wikieduonline Artifactory Universal Artifact Management Nexus Repository Software Component Management The Haskell Cabal Overview www haskell org Retrieved 2019 03 25 Maven Welcome to Apache Maven maven apache org Retrieved 2019 03 25 Julia Package Listing pkg julialang org Retrieved 2019 03 25 Quicklisp beta www quicklisp org Retrieved 2019 03 25 karann msft NuGet Package Manager UI Reference docs microsoft com Retrieved 2019 03 25 npm www npmjs com Retrieved 2019 03 25 Installing Perl Modules www cpan org www cpan org Retrieved 2019 03 25 Leisch Friedrich Creating R Packages A Tutorial PDF Graves Spencer B Dorai Raj Sundar Creating R Packages Using CRAN R Forge And Local R Archive Networks And Subversion SVN Repositories PDF The Comprehensive R Archive Network cran r project org Retrieved 2019 03 25 R Installation and Administration cran r project org Retrieved 2019 03 25 Wickham Hadley Bryan Jenny Package structure and state R Packages O Reilly Decan Alexandre Mens Tom Claes Maelick Grosjean Philippe 2015 On the Development and Distribution of R Packages An Empirical Analysis of the R Ecosystem Proceedings of the 2015 European Conference on Software Architecture Workshops ECSAW 15 1 6 doi 10 1145 2797433 2797476 S2CID 1680582 a b The Cargo Book Documentation Rust Programming Language Retrieved 2019 08 26 Rust Package Registry crates io Retrieved 2019 08 26 List of Top Repositories by Programming Language Stack Overflow Retrieved 2010 04 14 npm About www npmjs com Retrieved 2019 11 21 developers The pip pip The PyPA recommended tool for installing Python packages retrieved 2019 11 21 Apt Debian Wiki wiki debian org Retrieved 2019 11 22 Homebrew Homebrew Retrieved 2019 11 22 Yelp launches Yelp Fusion Microsoft creates Vcpkg tool and the new Touch Sense SDK for Android developers SD Times September 20 2016 Microsoft s C library manager now available for Linux and macOS SD Times April 25 2018 Chinthaguntla Keerthi Linux package management with YUM and RPM Enable Sysadmin Retrieved 2021 04 11 pacman ArchWiki wiki archlinux org Retrieved 2021 04 11 Chris Tucker 2007 03 15 Optimal Package Install Uninstall Manager PDF UC San Diego 1 Retrieved 2011 09 14 Cite journal requires journal help Linux repository classification schemes braintickle blogspot com 13 January 2006 Retrieved 2008 03 01 Retrieved from https en wikipedia org w index php title Software repository amp oldid 1051965716, wikipedia, wiki, book,

books

, library,

article

, read, download, free, free download, mp3, video, mp4, 3gp, jpg, jpeg, gif, png, picture, music, song, movie, book, game, games.