Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 mueller
|
574235e4cf |
+2
-2
@@ -1,3 +1,3 @@
|
||||
.*.swp
|
||||
# ---> Ansible
|
||||
*.retry
|
||||
vars_auth.yml
|
||||
|
||||
|
||||
@@ -0,0 +1,604 @@
|
||||
GNU GENERAL PUBLIC LICENSE
|
||||
Version 3, 29 June 2007
|
||||
|
||||
Copyright © 2007 Free Software Foundation, Inc. <http://fsf.org/>
|
||||
|
||||
Everyone is permitted to copy and distribute verbatim copies of this license
|
||||
document, but changing it is not allowed.
|
||||
|
||||
Preamble
|
||||
|
||||
The GNU General Public License is a free, copyleft license for software and
|
||||
other kinds of works.
|
||||
|
||||
The licenses for most software and other practical works are designed to take
|
||||
away your freedom to share and change the works. By contrast, the GNU General
|
||||
Public License is intended to guarantee your freedom to share and change all
|
||||
versions of a program--to make sure it remains free software for all its users.
|
||||
We, the Free Software Foundation, use the GNU General Public License for most
|
||||
of our software; it applies also to any other work released this way by its
|
||||
authors. You can apply it to your programs, too.
|
||||
|
||||
When we speak of free software, we are referring to freedom, not price. Our
|
||||
General Public Licenses are designed to make sure that you have the freedom
|
||||
to distribute copies of free software (and charge for them if you wish), that
|
||||
you receive source code or can get it if you want it, that you can change
|
||||
the software or use pieces of it in new free programs, and that you know you
|
||||
can do these things.
|
||||
|
||||
To protect your rights, we need to prevent others from denying you these rights
|
||||
or asking you to surrender the rights. Therefore, you have certain responsibilities
|
||||
if you distribute copies of the software, or if you modify it: responsibilities
|
||||
to respect the freedom of others.
|
||||
|
||||
For example, if you distribute copies of such a program, whether gratis or
|
||||
for a fee, you must pass on to the recipients the same freedoms that you received.
|
||||
You must make sure that they, too, receive or can get the source code. And
|
||||
you must show them these terms so they know their rights.
|
||||
|
||||
Developers that use the GNU GPL protect your rights with two steps: (1) assert
|
||||
copyright on the software, and (2) offer you this License giving you legal
|
||||
permission to copy, distribute and/or modify it.
|
||||
|
||||
For the developers' and authors' protection, the GPL clearly explains that
|
||||
there is no warranty for this free software. For both users' and authors'
|
||||
sake, the GPL requires that modified versions be marked as changed, so that
|
||||
their problems will not be attributed erroneously to authors of previous versions.
|
||||
|
||||
Some devices are designed to deny users access to install or run modified
|
||||
versions of the software inside them, although the manufacturer can do so.
|
||||
This is fundamentally incompatible with the aim of protecting users' freedom
|
||||
to change the software. The systematic pattern of such abuse occurs in the
|
||||
area of products for individuals to use, which is precisely where it is most
|
||||
unacceptable. Therefore, we have designed this version of the GPL to prohibit
|
||||
the practice for those products. If such problems arise substantially in other
|
||||
domains, we stand ready to extend this provision to those domains in future
|
||||
versions of the GPL, as needed to protect the freedom of users.
|
||||
|
||||
Finally, every program is threatened constantly by software patents. States
|
||||
should not allow patents to restrict development and use of software on general-purpose
|
||||
computers, but in those that do, we wish to avoid the special danger that
|
||||
patents applied to a free program could make it effectively proprietary. To
|
||||
prevent this, the GPL assures that patents cannot be used to render the program
|
||||
non-free.
|
||||
|
||||
The precise terms and conditions for copying, distribution and modification
|
||||
follow.
|
||||
|
||||
TERMS AND CONDITIONS
|
||||
|
||||
0. Definitions.
|
||||
|
||||
“This License” refers to version 3 of the GNU General Public License.
|
||||
|
||||
“Copyright” also means copyright-like laws that apply to other kinds of works,
|
||||
such as semiconductor masks.
|
||||
|
||||
“The Program” refers to any copyrightable work licensed under this License.
|
||||
Each licensee is addressed as “you”. “Licensees” and “recipients” may be individuals
|
||||
or organizations.
|
||||
|
||||
To “modify” a work means to copy from or adapt all or part of the work in
|
||||
a fashion requiring copyright permission, other than the making of an exact
|
||||
copy. The resulting work is called a “modified version” of the earlier work
|
||||
or a work “based on” the earlier work.
|
||||
|
||||
A “covered work” means either the unmodified Program or a work based on the
|
||||
Program.
|
||||
|
||||
To “propagate” a work means to do anything with it that, without permission,
|
||||
would make you directly or secondarily liable for infringement under applicable
|
||||
copyright law, except executing it on a computer or modifying a private copy.
|
||||
Propagation includes copying, distribution (with or without modification),
|
||||
making available to the public, and in some countries other activities as
|
||||
well.
|
||||
|
||||
To “convey” a work means any kind of propagation that enables other parties
|
||||
to make or receive copies. Mere interaction with a user through a computer
|
||||
network, with no transfer of a copy, is not conveying.
|
||||
|
||||
An interactive user interface displays “Appropriate Legal Notices” to the
|
||||
extent that it includes a convenient and prominently visible feature that
|
||||
(1) displays an appropriate copyright notice, and (2) tells the user that
|
||||
there is no warranty for the work (except to the extent that warranties are
|
||||
provided), that licensees may convey the work under this License, and how
|
||||
to view a copy of this License. If the interface presents a list of user commands
|
||||
or options, such as a menu, a prominent item in the list meets this criterion.
|
||||
|
||||
1. Source Code.
|
||||
The “source code” for a work means the preferred form of the work for making
|
||||
modifications to it. “Object code” means any non-source form of a work.
|
||||
|
||||
A “Standard Interface” means an interface that either is an official standard
|
||||
defined by a recognized standards body, or, in the case of interfaces specified
|
||||
for a particular programming language, one that is widely used among developers
|
||||
working in that language.
|
||||
|
||||
The “System Libraries” of an executable work include anything, other than
|
||||
the work as a whole, that (a) is included in the normal form of packaging
|
||||
a Major Component, but which is not part of that Major Component, and (b)
|
||||
serves only to enable use of the work with that Major Component, or to implement
|
||||
a Standard Interface for which an implementation is available to the public
|
||||
in source code form. A “Major Component”, in this context, means a major essential
|
||||
component (kernel, window system, and so on) of the specific operating system
|
||||
(if any) on which the executable work runs, or a compiler used to produce
|
||||
the work, or an object code interpreter used to run it.
|
||||
|
||||
The “Corresponding Source” for a work in object code form means all the source
|
||||
code needed to generate, install, and (for an executable work) run the object
|
||||
code and to modify the work, including scripts to control those activities.
|
||||
However, it does not include the work's System Libraries, or general-purpose
|
||||
tools or generally available free programs which are used unmodified in performing
|
||||
those activities but which are not part of the work. For example, Corresponding
|
||||
Source includes interface definition files associated with source files for
|
||||
the work, and the source code for shared libraries and dynamically linked
|
||||
subprograms that the work is specifically designed to require, such as by
|
||||
intimate data communication or control flow between those subprograms and
|
||||
other parts of the work.
|
||||
|
||||
The Corresponding Source need not include anything that users can regenerate
|
||||
automatically from other parts of the Corresponding Source.
|
||||
|
||||
The Corresponding Source for a work in source code form is that same work.
|
||||
|
||||
2. Basic Permissions.
|
||||
All rights granted under this License are granted for the term of copyright
|
||||
on the Program, and are irrevocable provided the stated conditions are met.
|
||||
This License explicitly affirms your unlimited permission to run the unmodified
|
||||
Program. The output from running a covered work is covered by this License
|
||||
only if the output, given its content, constitutes a covered work. This License
|
||||
acknowledges your rights of fair use or other equivalent, as provided by copyright
|
||||
law.
|
||||
|
||||
You may make, run and propagate covered works that you do not convey, without
|
||||
conditions so long as your license otherwise remains in force. You may convey
|
||||
covered works to others for the sole purpose of having them make modifications
|
||||
exclusively for you, or provide you with facilities for running those works,
|
||||
provided that you comply with the terms of this License in conveying all material
|
||||
for which you do not control copyright. Those thus making or running the covered
|
||||
works for you must do so exclusively on your behalf, under your direction
|
||||
and control, on terms that prohibit them from making any copies of your copyrighted
|
||||
material outside their relationship with you.
|
||||
|
||||
Conveying under any other circumstances is permitted solely under the conditions
|
||||
stated below. Sublicensing is not allowed; section 10 makes it unnecessary.
|
||||
|
||||
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||
No covered work shall be deemed part of an effective technological measure
|
||||
under any applicable law fulfilling obligations under article 11 of the WIPO
|
||||
copyright treaty adopted on 20 December 1996, or similar laws prohibiting
|
||||
or restricting circumvention of such measures.
|
||||
|
||||
When you convey a covered work, you waive any legal power to forbid circumvention
|
||||
of technological measures to the extent such circumvention is effected by
|
||||
exercising rights under this License with respect to the covered work, and
|
||||
you disclaim any intention to limit operation or modification of the work
|
||||
as a means of enforcing, against the work's users, your or third parties'
|
||||
legal rights to forbid circumvention of technological measures.
|
||||
|
||||
4. Conveying Verbatim Copies.
|
||||
You may convey verbatim copies of the Program's source code as you receive
|
||||
it, in any medium, provided that you conspicuously and appropriately publish
|
||||
on each copy an appropriate copyright notice; keep intact all notices stating
|
||||
that this License and any non-permissive terms added in accord with section
|
||||
7 apply to the code; keep intact all notices of the absence of any warranty;
|
||||
and give all recipients a copy of this License along with the Program.
|
||||
|
||||
You may charge any price or no price for each copy that you convey, and you
|
||||
may offer support or warranty protection for a fee.
|
||||
|
||||
5. Conveying Modified Source Versions.
|
||||
You may convey a work based on the Program, or the modifications to produce
|
||||
it from the Program, in the form of source code under the terms of section
|
||||
4, provided that you also meet all of these conditions:
|
||||
|
||||
a) The work must carry prominent notices stating that you modified it, and
|
||||
giving a relevant date.
|
||||
|
||||
b) The work must carry prominent notices stating that it is released under
|
||||
this License and any conditions added under section 7. This requirement modifies
|
||||
the requirement in section 4 to “keep intact all notices”.
|
||||
|
||||
c) You must license the entire work, as a whole, under this License to anyone
|
||||
who comes into possession of a copy. This License will therefore apply, along
|
||||
with any applicable section 7 additional terms, to the whole of the work,
|
||||
and all its parts, regardless of how they are packaged. This License gives
|
||||
no permission to license the work in any other way, but it does not invalidate
|
||||
such permission if you have separately received it.
|
||||
|
||||
d) If the work has interactive user interfaces, each must display Appropriate
|
||||
Legal Notices; however, if the Program has interactive interfaces that do
|
||||
not display Appropriate Legal Notices, your work need not make them do so.
|
||||
|
||||
A compilation of a covered work with other separate and independent works,
|
||||
which are not by their nature extensions of the covered work, and which are
|
||||
not combined with it such as to form a larger program, in or on a volume of
|
||||
a storage or distribution medium, is called an “aggregate” if the compilation
|
||||
and its resulting copyright are not used to limit the access or legal rights
|
||||
of the compilation's users beyond what the individual works permit. Inclusion
|
||||
of a covered work in an aggregate does not cause this License to apply to
|
||||
the other parts of the aggregate.
|
||||
|
||||
6. Conveying Non-Source Forms.
|
||||
You may convey a covered work in object code form under the terms of sections
|
||||
4 and 5, provided that you also convey the machine-readable Corresponding
|
||||
Source under the terms of this License, in one of these ways:
|
||||
|
||||
a) Convey the object code in, or embodied in, a physical product (including
|
||||
a physical distribution medium), accompanied by the Corresponding Source fixed
|
||||
on a durable physical medium customarily used for software interchange.
|
||||
|
||||
b) Convey the object code in, or embodied in, a physical product (including
|
||||
a physical distribution medium), accompanied by a written offer, valid for
|
||||
at least three years and valid for as long as you offer spare parts or customer
|
||||
support for that product model, to give anyone who possesses the object code
|
||||
either (1) a copy of the Corresponding Source for all the software in the
|
||||
product that is covered by this License, on a durable physical medium customarily
|
||||
used for software interchange, for a price no more than your reasonable cost
|
||||
of physically performing this conveying of source, or (2) access to copy the
|
||||
Corresponding Source from a network server at no charge.
|
||||
|
||||
c) Convey individual copies of the object code with a copy of the written
|
||||
offer to provide the Corresponding Source. This alternative is allowed only
|
||||
occasionally and noncommercially, and only if you received the object code
|
||||
with such an offer, in accord with subsection 6b.
|
||||
|
||||
d) Convey the object code by offering access from a designated place (gratis
|
||||
or for a charge), and offer equivalent access to the Corresponding Source
|
||||
in the same way through the same place at no further charge. You need not
|
||||
require recipients to copy the Corresponding Source along with the object
|
||||
code. If the place to copy the object code is a network server, the Corresponding
|
||||
Source may be on a different server (operated by you or a third party) that
|
||||
supports equivalent copying facilities, provided you maintain clear directions
|
||||
next to the object code saying where to find the Corresponding Source. Regardless
|
||||
of what server hosts the Corresponding Source, you remain obligated to ensure
|
||||
that it is available for as long as needed to satisfy these requirements.
|
||||
|
||||
e) Convey the object code using peer-to-peer transmission, provided you inform
|
||||
other peers where the object code and Corresponding Source of the work are
|
||||
being offered to the general public at no charge under subsection 6d.
|
||||
|
||||
A separable portion of the object code, whose source code is excluded from
|
||||
the Corresponding Source as a System Library, need not be included in conveying
|
||||
the object code work.
|
||||
|
||||
A “User Product” is either (1) a “consumer product”, which means any tangible
|
||||
personal property which is normally used for personal, family, or household
|
||||
purposes, or (2) anything designed or sold for incorporation into a dwelling.
|
||||
In determining whether a product is a consumer product, doubtful cases shall
|
||||
be resolved in favor of coverage. For a particular product received by a particular
|
||||
user, “normally used” refers to a typical or common use of that class of product,
|
||||
regardless of the status of the particular user or of the way in which the
|
||||
particular user actually uses, or expects or is expected to use, the product.
|
||||
A product is a consumer product regardless of whether the product has substantial
|
||||
commercial, industrial or non-consumer uses, unless such uses represent the
|
||||
only significant mode of use of the product.
|
||||
|
||||
“Installation Information” for a User Product means any methods, procedures,
|
||||
authorization keys, or other information required to install and execute modified
|
||||
versions of a covered work in that User Product from a modified version of
|
||||
its Corresponding Source. The information must suffice to ensure that the
|
||||
continued functioning of the modified object code is in no case prevented
|
||||
or interfered with solely because modification has been made.
|
||||
|
||||
If you convey an object code work under this section in, or with, or specifically
|
||||
for use in, a User Product, and the conveying occurs as part of a transaction
|
||||
in which the right of possession and use of the User Product is transferred
|
||||
to the recipient in perpetuity or for a fixed term (regardless of how the
|
||||
transaction is characterized), the Corresponding Source conveyed under this
|
||||
section must be accompanied by the Installation Information. But this requirement
|
||||
does not apply if neither you nor any third party retains the ability to install
|
||||
modified object code on the User Product (for example, the work has been installed
|
||||
in ROM).
|
||||
|
||||
The requirement to provide Installation Information does not include a requirement
|
||||
to continue to provide support service, warranty, or updates for a work that
|
||||
has been modified or installed by the recipient, or for the User Product in
|
||||
which it has been modified or installed. Access to a network may be denied
|
||||
when the modification itself materially and adversely affects the operation
|
||||
of the network or violates the rules and protocols for communication across
|
||||
the network.
|
||||
|
||||
Corresponding Source conveyed, and Installation Information provided, in accord
|
||||
with this section must be in a format that is publicly documented (and with
|
||||
an implementation available to the public in source code form), and must require
|
||||
no special password or key for unpacking, reading or copying.
|
||||
|
||||
7. Additional Terms.
|
||||
“Additional permissions” are terms that supplement the terms of this License
|
||||
by making exceptions from one or more of its conditions. Additional permissions
|
||||
that are applicable to the entire Program shall be treated as though they
|
||||
were included in this License, to the extent that they are valid under applicable
|
||||
law. If additional permissions apply only to part of the Program, that part
|
||||
may be used separately under those permissions, but the entire Program remains
|
||||
governed by this License without regard to the additional permissions.
|
||||
|
||||
When you convey a copy of a covered work, you may at your option remove any
|
||||
additional permissions from that copy, or from any part of it. (Additional
|
||||
permissions may be written to require their own removal in certain cases when
|
||||
you modify the work.) You may place additional permissions on material, added
|
||||
by you to a covered work, for which you have or can give appropriate copyright
|
||||
permission.
|
||||
|
||||
Notwithstanding any other provision of this License, for material you add
|
||||
to a covered work, you may (if authorized by the copyright holders of that
|
||||
material) supplement the terms of this License with terms:
|
||||
|
||||
a) Disclaiming warranty or limiting liability differently from the terms of
|
||||
sections 15 and 16 of this License; or
|
||||
|
||||
b) Requiring preservation of specified reasonable legal notices or author
|
||||
attributions in that material or in the Appropriate Legal Notices displayed
|
||||
by works containing it; or
|
||||
|
||||
c) Prohibiting misrepresentation of the origin of that material, or requiring
|
||||
that modified versions of such material be marked in reasonable ways as different
|
||||
from the original version; or
|
||||
|
||||
d) Limiting the use for publicity purposes of names of licensors or authors
|
||||
of the material; or
|
||||
|
||||
e) Declining to grant rights under trademark law for use of some trade names,
|
||||
trademarks, or service marks; or
|
||||
|
||||
f) Requiring indemnification of licensors and authors of that material by
|
||||
anyone who conveys the material (or modified versions of it) with contractual
|
||||
assumptions of liability to the recipient, for any liability that these contractual
|
||||
assumptions directly impose on those licensors and authors.
|
||||
|
||||
All other non-permissive additional terms are considered “further restrictions”
|
||||
within the meaning of section 10. If the Program as you received it, or any
|
||||
part of it, contains a notice stating that it is governed by this License
|
||||
along with a term that is a further restriction, you may remove that term.
|
||||
If a license document contains a further restriction but permits relicensing
|
||||
or conveying under this License, you may add to a covered work material governed
|
||||
by the terms of that license document, provided that the further restriction
|
||||
does not survive such relicensing or conveying.
|
||||
|
||||
If you add terms to a covered work in accord with this section, you must place,
|
||||
in the relevant source files, a statement of the additional terms that apply
|
||||
to those files, or a notice indicating where to find the applicable terms.
|
||||
|
||||
Additional terms, permissive or non-permissive, may be stated in the form
|
||||
of a separately written license, or stated as exceptions; the above requirements
|
||||
apply either way.
|
||||
|
||||
8. Termination.
|
||||
You may not propagate or modify a covered work except as expressly provided
|
||||
under this License. Any attempt otherwise to propagate or modify it is void,
|
||||
and will automatically terminate your rights under this License (including
|
||||
any patent licenses granted under the third paragraph of section 11).
|
||||
|
||||
However, if you cease all violation of this License, then your license from
|
||||
a particular copyright holder is reinstated (a) provisionally, unless and
|
||||
until the copyright holder explicitly and finally terminates your license,
|
||||
and (b) permanently, if the copyright holder fails to notify you of the violation
|
||||
by some reasonable means prior to 60 days after the cessation.
|
||||
|
||||
Moreover, your license from a particular copyright holder is reinstated permanently
|
||||
if the copyright holder notifies you of the violation by some reasonable means,
|
||||
this is the first time you have received notice of violation of this License
|
||||
(for any work) from that copyright holder, and you cure the violation prior
|
||||
to 30 days after your receipt of the notice.
|
||||
|
||||
Termination of your rights under this section does not terminate the licenses
|
||||
of parties who have received copies or rights from you under this License.
|
||||
If your rights have been terminated and not permanently reinstated, you do
|
||||
not qualify to receive new licenses for the same material under section 10.
|
||||
|
||||
9. Acceptance Not Required for Having Copies.
|
||||
You are not required to accept this License in order to receive or run a copy
|
||||
of the Program. Ancillary propagation of a covered work occurring solely as
|
||||
a consequence of using peer-to-peer transmission to receive a copy likewise
|
||||
does not require acceptance. However, nothing other than this License grants
|
||||
you permission to propagate or modify any covered work. These actions infringe
|
||||
copyright if you do not accept this License. Therefore, by modifying or propagating
|
||||
a covered work, you indicate your acceptance of this License to do so.
|
||||
|
||||
10. Automatic Licensing of Downstream Recipients.
|
||||
Each time you convey a covered work, the recipient automatically receives
|
||||
a license from the original licensors, to run, modify and propagate that work,
|
||||
subject to this License. You are not responsible for enforcing compliance
|
||||
by third parties with this License.
|
||||
|
||||
An “entity transaction” is a transaction transferring control of an organization,
|
||||
or substantially all assets of one, or subdividing an organization, or merging
|
||||
organizations. If propagation of a covered work results from an entity transaction,
|
||||
each party to that transaction who receives a copy of the work also receives
|
||||
whatever licenses to the work the party's predecessor in interest had or could
|
||||
give under the previous paragraph, plus a right to possession of the Corresponding
|
||||
Source of the work from the predecessor in interest, if the predecessor has
|
||||
it or can get it with reasonable efforts.
|
||||
|
||||
You may not impose any further restrictions on the exercise of the rights
|
||||
granted or affirmed under this License. For example, you may not impose a
|
||||
license fee, royalty, or other charge for exercise of rights granted under
|
||||
this License, and you may not initiate litigation (including a cross-claim
|
||||
or counterclaim in a lawsuit) alleging that any patent claim is infringed
|
||||
by making, using, selling, offering for sale, or importing the Program or
|
||||
any portion of it.
|
||||
|
||||
11. Patents.
|
||||
A “contributor” is a copyright holder who authorizes use under this License
|
||||
of the Program or a work on which the Program is based. The work thus licensed
|
||||
is called the contributor's “contributor version”.
|
||||
|
||||
A contributor's “essential patent claims” are all patent claims owned or controlled
|
||||
by the contributor, whether already acquired or hereafter acquired, that would
|
||||
be infringed by some manner, permitted by this License, of making, using,
|
||||
or selling its contributor version, but do not include claims that would be
|
||||
infringed only as a consequence of further modification of the contributor
|
||||
version. For purposes of this definition, “control” includes the right to
|
||||
grant patent sublicenses in a manner consistent with the requirements of this
|
||||
License.
|
||||
|
||||
Each contributor grants you a non-exclusive, worldwide, royalty-free patent
|
||||
license under the contributor's essential patent claims, to make, use, sell,
|
||||
offer for sale, import and otherwise run, modify and propagate the contents
|
||||
of its contributor version.
|
||||
|
||||
In the following three paragraphs, a “patent license” is any express agreement
|
||||
or commitment, however denominated, not to enforce a patent (such as an express
|
||||
permission to practice a patent or covenant not to sue for patent infringement).
|
||||
To “grant” such a patent license to a party means to make such an agreement
|
||||
or commitment not to enforce a patent against the party.
|
||||
|
||||
If you convey a covered work, knowingly relying on a patent license, and the
|
||||
Corresponding Source of the work is not available for anyone to copy, free
|
||||
of charge and under the terms of this License, through a publicly available
|
||||
network server or other readily accessible means, then you must either (1)
|
||||
cause the Corresponding Source to be so available, or (2) arrange to deprive
|
||||
yourself of the benefit of the patent license for this particular work, or
|
||||
(3) arrange, in a manner consistent with the requirements of this License,
|
||||
to extend the patent license to downstream recipients. “Knowingly relying”
|
||||
means you have actual knowledge that, but for the patent license, your conveying
|
||||
the covered work in a country, or your recipient's use of the covered work
|
||||
in a country, would infringe one or more identifiable patents in that country
|
||||
that you have reason to believe are valid.
|
||||
|
||||
If, pursuant to or in connection with a single transaction or arrangement,
|
||||
you convey, or propagate by procuring conveyance of, a covered work, and grant
|
||||
a patent license to some of the parties receiving the covered work authorizing
|
||||
them to use, propagate, modify or convey a specific copy of the covered work,
|
||||
then the patent license you grant is automatically extended to all recipients
|
||||
of the covered work and works based on it.
|
||||
|
||||
A patent license is “discriminatory” if it does not include within the scope
|
||||
of its coverage, prohibits the exercise of, or is conditioned on the non-exercise
|
||||
of one or more of the rights that are specifically granted under this License.
|
||||
You may not convey a covered work if you are a party to an arrangement with
|
||||
a third party that is in the business of distributing software, under which
|
||||
you make payment to the third party based on the extent of your activity of
|
||||
conveying the work, and under which the third party grants, to any of the
|
||||
parties who would receive the covered work from you, a discriminatory patent
|
||||
license (a) in connection with copies of the covered work conveyed by you
|
||||
(or copies made from those copies), or (b) primarily for and in connection
|
||||
with specific products or compilations that contain the covered work, unless
|
||||
you entered into that arrangement, or that patent license was granted, prior
|
||||
to 28 March 2007.
|
||||
|
||||
Nothing in this License shall be construed as excluding or limiting any implied
|
||||
license or other defenses to infringement that may otherwise be available
|
||||
to you under applicable patent law.
|
||||
|
||||
12. No Surrender of Others' Freedom.
|
||||
If conditions are imposed on you (whether by court order, agreement or otherwise)
|
||||
that contradict the conditions of this License, they do not excuse you from
|
||||
the conditions of this License. If you cannot convey a covered work so as
|
||||
to satisfy simultaneously your obligations under this License and any other
|
||||
pertinent obligations, then as a consequence you may not convey it at all.
|
||||
For example, if you agree to terms that obligate you to collect a royalty
|
||||
for further conveying from those to whom you convey the Program, the only
|
||||
way you could satisfy both those terms and this License would be to refrain
|
||||
entirely from conveying the Program.
|
||||
|
||||
13. Use with the GNU Affero General Public License.
|
||||
Notwithstanding any other provision of this License, you have permission to
|
||||
link or combine any covered work with a work licensed under version 3 of the
|
||||
GNU Affero General Public License into a single combined work, and to convey
|
||||
the resulting work. The terms of this License will continue to apply to the
|
||||
part which is the covered work, but the special requirements of the GNU Affero
|
||||
General Public License, section 13, concerning interaction through a network
|
||||
will apply to the combination as such.
|
||||
|
||||
14. Revised Versions of this License.
|
||||
The Free Software Foundation may publish revised and/or new versions of the
|
||||
GNU General Public License from time to time. Such new versions will be similar
|
||||
in spirit to the present version, but may differ in detail to address new
|
||||
problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the Program specifies
|
||||
that a certain numbered version of the GNU General Public License “or any
|
||||
later version” applies to it, you have the option of following the terms and
|
||||
conditions either of that numbered version or of any later version published
|
||||
by the Free Software Foundation. If the Program does not specify a version
|
||||
number of the GNU General Public License, you may choose any version ever
|
||||
published by the Free Software Foundation.
|
||||
|
||||
If the Program specifies that a proxy can decide which future versions of
|
||||
the GNU General Public License can be used, that proxy's public statement
|
||||
of acceptance of a version permanently authorizes you to choose that version
|
||||
for the Program.
|
||||
|
||||
Later license versions may give you additional or different permissions. However,
|
||||
no additional obligations are imposed on any author or copyright holder as
|
||||
a result of your choosing to follow a later version.
|
||||
|
||||
15. Disclaimer of Warranty.
|
||||
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE
|
||||
LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
|
||||
OTHER PARTIES PROVIDE THE PROGRAM “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER
|
||||
EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
|
||||
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
|
||||
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM
|
||||
PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR
|
||||
CORRECTION.
|
||||
|
||||
16. Limitation of Liability.
|
||||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL
|
||||
ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM
|
||||
AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL,
|
||||
INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO
|
||||
USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED
|
||||
INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE
|
||||
PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER
|
||||
PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
|
||||
|
||||
17. Interpretation of Sections 15 and 16.
|
||||
If the disclaimer of warranty and limitation of liability provided above cannot
|
||||
be given local legal effect according to their terms, reviewing courts shall
|
||||
apply local law that most closely approximates an absolute waiver of all civil
|
||||
liability in connection with the Program, unless a warranty or assumption
|
||||
of liability accompanies a copy of the Program in return for a fee.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
How to Apply These Terms to Your New Programs
|
||||
|
||||
If you develop a new program, and you want it to be of the greatest possible
|
||||
use to the public, the best way to achieve this is to make it free software
|
||||
which everyone can redistribute and change under these terms.
|
||||
|
||||
To do so, attach the following notices to the program. It is safest to attach
|
||||
them to the start of each source file to most effectively state the exclusion
|
||||
of warranty; and each file should have at least the “copyright” line and a
|
||||
pointer to where the full notice is found.
|
||||
|
||||
<one line to give the program's name and a brief idea of what it does.>
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
This program is free software: you can redistribute it and/or modify it under
|
||||
the terms of the GNU General Public License as published by the Free Software
|
||||
Foundation, either version 3 of the License, or (at your option) any later
|
||||
version.
|
||||
|
||||
This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU General Public License along with
|
||||
this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
If the program does terminal interaction, make it output a short notice like
|
||||
this when it starts in an interactive mode:
|
||||
|
||||
<program> Copyright (C) <year> <name of author>
|
||||
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
|
||||
This is free software, and you are welcome to redistribute it under certain
|
||||
conditions; type `show c' for details.
|
||||
|
||||
The hypothetical commands `show w' and `show c' should show the appropriate
|
||||
parts of the General Public License. Of course, your program's commands might
|
||||
be different; for a GUI interface, you would use an “about box”.
|
||||
|
||||
You should also get your employer (if you work as a programmer) or school,
|
||||
if any, to sign a “copyright disclaimer” for the program, if necessary. For
|
||||
more information on this, and how to apply and follow the GNU GPL, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
The GNU General Public License does not permit incorporating your program
|
||||
into proprietary programs. If your program is a subroutine library, you may
|
||||
consider it more useful to permit linking proprietary applications with the
|
||||
library. If this is what you want to do, use the GNU Lesser General Public
|
||||
License instead of this License. But first, please read <http://www.gnu.org/philosophy/why-not-lgpl.html>.
|
||||
@@ -1,19 +1,3 @@
|
||||
# Requirements:
|
||||
CentOS 7 vm equipped with 2 CPU cores and 2 disks: one main disk containing OS
|
||||
and an additional virtio disk containing docker images. Typically choose 5GB
|
||||
for size of each disk.
|
||||
# cobald-testenv
|
||||
|
||||
# Tags
|
||||
- htcondor, singularity, docker, slurm
|
||||
- htcondor-containered, slurm-config
|
||||
|
||||
Just update slurm config using --tags slurm-config.
|
||||
Full setup of just slurm using --skip-tags htcondor,singularity
|
||||
|
||||
# Example: Setup SLURM
|
||||
First adjust inv.yml to you needs, specifying the hostname of your VM. Run:
|
||||
|
||||
ansible-playbook -i inv.yml --skip-tags htcondor,singularity play.yml
|
||||
|
||||
Configuration goes into ```/container/volumes/slurm/slurm.conf``` generated
|
||||
from ```roles/slurm/templates/slurm.conf.j2```.
|
||||
WIP. htcondor, slurm and cobald/tardis test environment
|
||||
@@ -1,27 +0,0 @@
|
||||
---
|
||||
- hosts: all
|
||||
tasks:
|
||||
- name: "install epel repo" # for htop etc.
|
||||
yum:
|
||||
name: epel-release
|
||||
state: present
|
||||
|
||||
- name: "install tools"
|
||||
yum:
|
||||
name: [ vim-enhanced, htop, screen, bind-utils, nmap-ncat, net-tools ]
|
||||
state: present
|
||||
|
||||
- name: "screenrc native scrolling in tmux"
|
||||
copy:
|
||||
content: "termcapinfo xterm* ti@:te@\ntermcapinfo screen* ti@:te@\n"
|
||||
dest: "{{item}}"
|
||||
with_items:
|
||||
- "~{{unpriv_user}}/.screenrc"
|
||||
- "~root/.screenrc"
|
||||
|
||||
- name: "install ssh-key"
|
||||
authorized_key:
|
||||
user: "{{cfg_unpriv_user}}"
|
||||
key: "{{cfg_ssh_key}}"
|
||||
state: present
|
||||
|
||||
-61
@@ -1,61 +0,0 @@
|
||||
---
|
||||
- hosts: slurm, cobald
|
||||
vars:
|
||||
container_privileged: True
|
||||
slurm_num_nodes: 10
|
||||
tasks:
|
||||
- name: "setup docker"
|
||||
import_role: name=docker
|
||||
tags: docker
|
||||
|
||||
- name: "get facts from existing cobald instance (i.e. hostname)"
|
||||
include_role:
|
||||
name: cobald
|
||||
tasks_from: facts
|
||||
apply:
|
||||
tags: slurm, cobald, slurm-config
|
||||
tags: slurm, cobald, slurm-config
|
||||
vars:
|
||||
container_name: cobald
|
||||
|
||||
- name: "setup slurm test environment in docker containers"
|
||||
include_role:
|
||||
name: slurm
|
||||
apply:
|
||||
tags: slurm
|
||||
vars:
|
||||
slurm_user: slurm # or root
|
||||
slurm_user_accounts:
|
||||
- name: cobald
|
||||
dir: /var/lib/cobald
|
||||
num_nodes: "{{slurm_num_nodes}}"
|
||||
extra_nodes:
|
||||
- name: cobald
|
||||
hostname: "{{cobald_container_hostname}}" # from cobald/facts.yml above
|
||||
# hostname is used as NodeHostname, which is used slurms "networking
|
||||
# code" (https://bugs.schedmd.com/show_bug.cgi?id=8615).
|
||||
# It works either way around, but one of NodeName or NodeHostname has
|
||||
# to match the container name (-n flag, not --hostname) since when
|
||||
# submitting tasks to the slurm controller, it matches access
|
||||
# permissions against a reverse lookup of the submitting ip address.
|
||||
# Docker always and unconfigureably resolves the container ip in any
|
||||
# network to containername.netname, where containername is the
|
||||
# containers runtime name (not hostname supplied!) and netname is
|
||||
# the network name in host environment. We should run our own dns...
|
||||
docker_network: slurm
|
||||
slurm_hostsystem_cluster_access: True
|
||||
when: '"slurm" in group_names'
|
||||
tags: slurm, cobald, influxdb, slurm-config
|
||||
# tags: cobald requires some slurm facts, so cobald tag is included here
|
||||
|
||||
- name: "install cobald"
|
||||
include_role:
|
||||
name: cobald
|
||||
apply:
|
||||
tags: cobald
|
||||
vars:
|
||||
cobald_slurm: True
|
||||
container_name: cobald
|
||||
# docker_network: slurm # overriden by vars/slurm.yml
|
||||
when: '"cobald" in group_names'
|
||||
tags: cobald, influxdb, singularity
|
||||
@@ -1,32 +0,0 @@
|
||||
---
|
||||
- hosts: htcondor
|
||||
tasks:
|
||||
- name: "install htcondor repo"
|
||||
yum:
|
||||
name: https://research.cs.wisc.edu/htcondor/repo/8.9/htcondor-release-current.el7.noarch.rpm
|
||||
state: present
|
||||
tags: htcondor
|
||||
|
||||
- name: "install htcondor software "
|
||||
yum:
|
||||
name: htcondor-ce
|
||||
state: present
|
||||
tags: htcondor
|
||||
|
||||
- name: "remove minicondor configuration"
|
||||
yum:
|
||||
name: minicondor
|
||||
state: absent
|
||||
tags: htcondor
|
||||
|
||||
- name: "setup singularity"
|
||||
import_role: name="singularity"
|
||||
tags: singularity
|
||||
|
||||
- name: "setup docker"
|
||||
import_role: name=docker
|
||||
tags: docker
|
||||
|
||||
- name: "setup htcondor test environment in docker containers"
|
||||
import_role: name=docker-htcondor
|
||||
tags: htcondor-containered, htcondor
|
||||
@@ -1,4 +0,0 @@
|
||||
mkdir -p ./collections/ansible_collections/community
|
||||
# git clone --depth=1 -b 1.2.1 https://github.com/ansible-collections/community.grafana.git ./collections/ansible_collections/community/grafana
|
||||
git clone --depth=1 -b 1.2.1-extended https://github.com/ansible_community.grafana.git ./collections/ansible_collections/community/grafana
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
all:
|
||||
hosts:
|
||||
ed-c7-1:
|
||||
ansible_user: root
|
||||
ansible_host: ed-c7-1.virt.magni.thoto.net
|
||||
ssh_args: -o ControlMaster=auto -o ControlPersist=60s
|
||||
# ansible_host: 192.168.122.139
|
||||
unpriv_user: thoto
|
||||
cfg_unpriv_user: thoto
|
||||
ed-c7-2:
|
||||
ansible_user: root
|
||||
ansible_host: ed-c7-2.virt.uller.thoto.net
|
||||
# ansible_host: 192.168.123.60 # + jumphost
|
||||
ssh_args: -o ControlMaster=auto -o ControlPersist=60s
|
||||
unpriv_user: thoto
|
||||
cfg_unpriv_user: thoto
|
||||
children:
|
||||
htcondor:
|
||||
hosts:
|
||||
ed-c7-1:
|
||||
slurm:
|
||||
hosts:
|
||||
ed-c7-1:
|
||||
ed-c7-2:
|
||||
cobald:
|
||||
hosts:
|
||||
ed-c7-1:
|
||||
ed-c7-2:
|
||||
@@ -1,10 +0,0 @@
|
||||
---
|
||||
- name: base setup
|
||||
import_playbook: base.yml
|
||||
|
||||
- name: setup htcondor
|
||||
import_playbook: htcondor.yml
|
||||
when: '"htcondor" in group_names'
|
||||
|
||||
- name: setup slurm and cobald
|
||||
import_playbook: cobald.yml
|
||||
@@ -1,6 +0,0 @@
|
||||
cobald_domainname: cobald.local
|
||||
influx_admin_user: my-user
|
||||
influx_admin_pw: my-password
|
||||
influx_org: my-org
|
||||
influx_pubport: 28086
|
||||
influx_bucket: batleth
|
||||
@@ -1,3 +0,0 @@
|
||||
#!/bin/sh
|
||||
[ /slurm-singimage/slurmd.sif -nt /shared/slurmd.sif ] && \
|
||||
cp /slurm-singimage/slurmd.sif /shared/slurmd.sif
|
||||
@@ -1,3 +0,0 @@
|
||||
#!/bin/sh
|
||||
slurmd --conf-server ${slurmctld} -D -N ${nodename} 2>/dev/null 1>/dev/null &
|
||||
tail --retry --pid $! -f ${SLURMD_LOG_PATH} ${SLURM_SCHED_LOG_PATH}
|
||||
@@ -1,11 +0,0 @@
|
||||
###
|
||||
#
|
||||
# Slurm cgroup support configuration file
|
||||
#
|
||||
# See man slurm.conf and man cgroup.conf for further
|
||||
# information on cgroup configuration parameters
|
||||
#--
|
||||
CgroupAutomount=no
|
||||
|
||||
ConstrainCores=no
|
||||
ConstrainRAMSpace=no
|
||||
@@ -1,31 +0,0 @@
|
||||
---
|
||||
pipeline:
|
||||
- __type__: cobald.controller.linear.LinearController
|
||||
low_utilisation: 0.9
|
||||
high_allocation: 0.9
|
||||
rate: 0.10
|
||||
- !Limiter
|
||||
minimum: 3
|
||||
- !TelegrafPipelineMonitor
|
||||
poll: True
|
||||
- !TardisPoolFactory
|
||||
configuration: /etc/cobald/tardis.yaml
|
||||
logging:
|
||||
version: 1
|
||||
root:
|
||||
level: DEBUG
|
||||
handlers: [console, file]
|
||||
handlers:
|
||||
console:
|
||||
class: logging.StreamHandler
|
||||
formatter: test
|
||||
level: DEBUG
|
||||
stream: ext://sys.stderr
|
||||
file:
|
||||
class: logging.handlers.RotatingFileHandler
|
||||
formatter: test
|
||||
level: WARNING
|
||||
filename: /var/log/cobald/cobald-tardis.log
|
||||
formatters:
|
||||
test:
|
||||
format: " %(name)s %(message)s"
|
||||
@@ -1,41 +0,0 @@
|
||||
Plugins:
|
||||
SqliteRegistry:
|
||||
db_file: /tmp/drone_registry.db
|
||||
TelegrafMonitoring:
|
||||
host: ed-telegraf
|
||||
port: 8094
|
||||
#BatchSystem:
|
||||
# adapter: FakeBatchSystem
|
||||
# allocation: 1.0
|
||||
# utilisation: !PeriodicValue
|
||||
# period: 60
|
||||
# amplitude: 0.15
|
||||
# offset: 0.80
|
||||
## phase: 1.
|
||||
# phase: 1.6
|
||||
# machine_status: Available
|
||||
BatchSystem:
|
||||
adapter: Slurm
|
||||
max_age: 0.1
|
||||
options:
|
||||
partition: cobald
|
||||
Sites:
|
||||
- name: slurmtest
|
||||
adapter: Slurm
|
||||
quota: 20
|
||||
slurmtest:
|
||||
# executor: ...
|
||||
StatusUpdate: 0.1
|
||||
MachineTypes:
|
||||
- m1.a
|
||||
MachineTypeConfiguration:
|
||||
m1.a:
|
||||
Walltime: 5
|
||||
Partition: container
|
||||
StartupCommand: /usr/local/bin/start-drone
|
||||
# SubmitOptions: ...
|
||||
MachineMetaData:
|
||||
m1.a:
|
||||
Cores: 3 # cores
|
||||
Memory: 1 # GB
|
||||
Disk: 4 # not passed
|
||||
@@ -1,7 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
for i in /usr/local/lib/entrypoints.d/* ; do
|
||||
[ -f $i ] && /bin/sh $i || break
|
||||
done
|
||||
|
||||
exec "${@:-/bin/bash}"
|
||||
@@ -1,589 +0,0 @@
|
||||
{
|
||||
"annotations": {
|
||||
"list": [
|
||||
{
|
||||
"builtIn": 1,
|
||||
"datasource": "-- Grafana --",
|
||||
"enable": true,
|
||||
"hide": true,
|
||||
"iconColor": "rgba(0, 211, 255, 1)",
|
||||
"name": "Annotations & Alerts",
|
||||
"type": "dashboard"
|
||||
}
|
||||
]
|
||||
},
|
||||
"editable": true,
|
||||
"gnetId": null,
|
||||
"graphTooltip": 0,
|
||||
"id": 1,
|
||||
"iteration": 1623317629899,
|
||||
"links": [],
|
||||
"panels": [
|
||||
{
|
||||
"collapsed": false,
|
||||
"datasource": null,
|
||||
"gridPos": {
|
||||
"h": 1,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 0
|
||||
},
|
||||
"id": 4,
|
||||
"panels": [],
|
||||
"title": "Row title",
|
||||
"type": "row"
|
||||
},
|
||||
{
|
||||
"aliasColors": {},
|
||||
"bars": false,
|
||||
"dashLength": 10,
|
||||
"dashes": false,
|
||||
"datasource": "InfluxDB",
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "none"
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"fill": 1,
|
||||
"fillGradient": 0,
|
||||
"gridPos": {
|
||||
"h": 9,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 1
|
||||
},
|
||||
"hiddenSeries": false,
|
||||
"id": 2,
|
||||
"interval": "1s",
|
||||
"legend": {
|
||||
"avg": false,
|
||||
"current": false,
|
||||
"max": true,
|
||||
"min": true,
|
||||
"show": true,
|
||||
"total": false,
|
||||
"values": true
|
||||
},
|
||||
"lines": true,
|
||||
"linewidth": 1,
|
||||
"maxDataPoints": 200,
|
||||
"nullPointMode": "null",
|
||||
"options": {
|
||||
"alertThreshold": true
|
||||
},
|
||||
"percentage": false,
|
||||
"pluginVersion": "7.5.7",
|
||||
"pointradius": 2,
|
||||
"points": false,
|
||||
"renderer": "flot",
|
||||
"seriesOverrides": [],
|
||||
"spaceLength": 10,
|
||||
"stack": false,
|
||||
"steppedLine": false,
|
||||
"targets": [
|
||||
{
|
||||
"groupBy": [
|
||||
{
|
||||
"params": [
|
||||
"$__interval"
|
||||
],
|
||||
"type": "time"
|
||||
},
|
||||
{
|
||||
"params": [
|
||||
"null"
|
||||
],
|
||||
"type": "fill"
|
||||
}
|
||||
],
|
||||
"orderByTime": "ASC",
|
||||
"policy": "default",
|
||||
"query": "f_r = (r, accumulator) => ({\n _value: accumulator._value + (\n if r._value == \"AvailableState\" then 1\n else if r._value == \"DownState\" then -1\n else 0)\n })\n\nnodes = from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> keep(columns: [\"tardis_machine_name\", \"_time\"])\n |> sort(columns: [\"_time\"], desc: true)\n |> unique(column: \"tardis_machine_name\")\n// |> yield()\n\noffset = from(bucket: \"batleth\")\n |> range(start: 0, stop: v.timeRangeStart)\n |> filter(fn: (r) => r._field == \"state\")\n |> group(columns: [\"tardis_machine_name\", \"machine_type\"])\n |> reduce(fn: f_r, identity: {_value: 0})\n |> duplicate(column: \"_stop\", as: \"_time\")\n\nnew = from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._field == \"state\")\n// |> filter(fn: (r) => r.tardis_machine_name == \"${machine}\")\n |> group(columns: [\"tardis_machine_name\", \"machine_type\"])\n |> window(every: $__interval)\n |> reduce(fn: f_r, identity: {_value: 0})\n |> duplicate(column: \"_stop\", as: \"_time\")\n\nunion(tables: [offset, new])\n |> window(every: inf)\n |> cumulativeSum()\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> yield()\n",
|
||||
"refId": "A",
|
||||
"resultFormat": "time_series",
|
||||
"select": [
|
||||
[
|
||||
{
|
||||
"params": [
|
||||
"value"
|
||||
],
|
||||
"type": "field"
|
||||
},
|
||||
{
|
||||
"params": [],
|
||||
"type": "mean"
|
||||
}
|
||||
]
|
||||
],
|
||||
"tags": []
|
||||
}
|
||||
],
|
||||
"thresholds": [],
|
||||
"timeFrom": null,
|
||||
"timeRegions": [],
|
||||
"timeShift": null,
|
||||
"title": "nodes running",
|
||||
"tooltip": {
|
||||
"shared": true,
|
||||
"sort": 0,
|
||||
"value_type": "individual"
|
||||
},
|
||||
"type": "graph",
|
||||
"xaxis": {
|
||||
"buckets": null,
|
||||
"mode": "time",
|
||||
"name": null,
|
||||
"show": true,
|
||||
"values": []
|
||||
},
|
||||
"yaxes": [
|
||||
{
|
||||
"format": "none",
|
||||
"label": null,
|
||||
"logBase": 1,
|
||||
"max": null,
|
||||
"min": null,
|
||||
"show": true
|
||||
},
|
||||
{
|
||||
"format": "short",
|
||||
"label": null,
|
||||
"logBase": 1,
|
||||
"max": null,
|
||||
"min": null,
|
||||
"show": true
|
||||
}
|
||||
],
|
||||
"yaxis": {
|
||||
"align": false,
|
||||
"alignLevel": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"aliasColors": {},
|
||||
"bars": false,
|
||||
"dashLength": 10,
|
||||
"dashes": false,
|
||||
"datasource": "InfluxDB",
|
||||
"fieldConfig": {
|
||||
"defaults": {},
|
||||
"overrides": []
|
||||
},
|
||||
"fill": 1,
|
||||
"fillGradient": 0,
|
||||
"gridPos": {
|
||||
"h": 9,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 1
|
||||
},
|
||||
"hiddenSeries": false,
|
||||
"id": 8,
|
||||
"interval": "1s",
|
||||
"legend": {
|
||||
"avg": false,
|
||||
"current": false,
|
||||
"max": false,
|
||||
"min": false,
|
||||
"show": true,
|
||||
"total": false,
|
||||
"values": false
|
||||
},
|
||||
"lines": true,
|
||||
"linewidth": 1,
|
||||
"maxDataPoints": null,
|
||||
"nullPointMode": "null",
|
||||
"options": {
|
||||
"alertThreshold": true
|
||||
},
|
||||
"percentage": false,
|
||||
"pluginVersion": "7.5.7",
|
||||
"pointradius": 2,
|
||||
"points": false,
|
||||
"renderer": "flot",
|
||||
"seriesOverrides": [],
|
||||
"spaceLength": 10,
|
||||
"stack": false,
|
||||
"steppedLine": false,
|
||||
"targets": [
|
||||
{
|
||||
"groupBy": [
|
||||
{
|
||||
"params": [
|
||||
"$__interval"
|
||||
],
|
||||
"type": "time"
|
||||
},
|
||||
{
|
||||
"params": [
|
||||
"null"
|
||||
],
|
||||
"type": "fill"
|
||||
}
|
||||
],
|
||||
"orderByTime": "ASC",
|
||||
"policy": "default",
|
||||
"query": "from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"tardis_pipeline\")\n |> filter(fn: (r) => r._field == \"demand\" or r._field == \"supply\")\n |> drop(columns: [\"host\"])\n |> aggregateWindow(every: $__interval, fn: mean)\n |> yield()",
|
||||
"queryType": "randomWalk",
|
||||
"refId": "A",
|
||||
"resultFormat": "time_series",
|
||||
"select": [
|
||||
[
|
||||
{
|
||||
"params": [
|
||||
"value"
|
||||
],
|
||||
"type": "field"
|
||||
},
|
||||
{
|
||||
"params": [],
|
||||
"type": "mean"
|
||||
}
|
||||
]
|
||||
],
|
||||
"tags": []
|
||||
}
|
||||
],
|
||||
"thresholds": [],
|
||||
"timeFrom": null,
|
||||
"timeRegions": [],
|
||||
"timeShift": null,
|
||||
"title": "pipeline demand/supply (mean)",
|
||||
"tooltip": {
|
||||
"shared": true,
|
||||
"sort": 0,
|
||||
"value_type": "individual"
|
||||
},
|
||||
"type": "graph",
|
||||
"xaxis": {
|
||||
"buckets": null,
|
||||
"mode": "time",
|
||||
"name": null,
|
||||
"show": true,
|
||||
"values": []
|
||||
},
|
||||
"yaxes": [
|
||||
{
|
||||
"$$hashKey": "object:115",
|
||||
"decimals": null,
|
||||
"format": "short",
|
||||
"label": "cpus",
|
||||
"logBase": 1,
|
||||
"max": null,
|
||||
"min": null,
|
||||
"show": true
|
||||
},
|
||||
{
|
||||
"$$hashKey": "object:116",
|
||||
"format": "short",
|
||||
"label": null,
|
||||
"logBase": 1,
|
||||
"max": null,
|
||||
"min": null,
|
||||
"show": true
|
||||
}
|
||||
],
|
||||
"yaxis": {
|
||||
"align": false,
|
||||
"alignLevel": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"aliasColors": {},
|
||||
"bars": false,
|
||||
"dashLength": 10,
|
||||
"dashes": false,
|
||||
"datasource": "InfluxDB",
|
||||
"fieldConfig": {
|
||||
"defaults": {},
|
||||
"overrides": []
|
||||
},
|
||||
"fill": 1,
|
||||
"fillGradient": 0,
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 10
|
||||
},
|
||||
"hiddenSeries": false,
|
||||
"id": 6,
|
||||
"interval": "1s",
|
||||
"legend": {
|
||||
"avg": false,
|
||||
"current": false,
|
||||
"max": false,
|
||||
"min": false,
|
||||
"show": true,
|
||||
"total": false,
|
||||
"values": false
|
||||
},
|
||||
"lines": true,
|
||||
"linewidth": 1,
|
||||
"nullPointMode": "null",
|
||||
"options": {
|
||||
"alertThreshold": true
|
||||
},
|
||||
"percentage": false,
|
||||
"pluginVersion": "7.5.7",
|
||||
"pointradius": 2,
|
||||
"points": false,
|
||||
"renderer": "flot",
|
||||
"seriesOverrides": [],
|
||||
"spaceLength": 10,
|
||||
"stack": false,
|
||||
"steppedLine": false,
|
||||
"targets": [
|
||||
{
|
||||
"groupBy": [
|
||||
{
|
||||
"params": [
|
||||
"$__interval"
|
||||
],
|
||||
"type": "time"
|
||||
},
|
||||
{
|
||||
"params": [
|
||||
"null"
|
||||
],
|
||||
"type": "fill"
|
||||
}
|
||||
],
|
||||
"orderByTime": "ASC",
|
||||
"policy": "default",
|
||||
"query": "f = (r, accumulator) => ({\n _value: accumulator._value + (if r._value == \"AvailableState\" then 1 else if r._value == \"DownState\" then -1 else 0)\n })\n\nfrom(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._field == \"state\")\n |> group(columns: [\"tardis_machine_name\"])\n |> window(every: $__interval)\n |> reduce(fn: f, identity: {_value: 0})\n |> duplicate(column: \"_stop\", as: \"_time\")\n |> window(every: inf, timeColumn: \"_time\")\n |> yield()",
|
||||
"queryType": "randomWalk",
|
||||
"refId": "A",
|
||||
"resultFormat": "time_series",
|
||||
"select": [
|
||||
[
|
||||
{
|
||||
"params": [
|
||||
"value"
|
||||
],
|
||||
"type": "field"
|
||||
},
|
||||
{
|
||||
"params": [],
|
||||
"type": "mean"
|
||||
}
|
||||
]
|
||||
],
|
||||
"tags": []
|
||||
}
|
||||
],
|
||||
"thresholds": [],
|
||||
"timeFrom": null,
|
||||
"timeRegions": [],
|
||||
"timeShift": null,
|
||||
"title": "node fluctuation",
|
||||
"tooltip": {
|
||||
"shared": true,
|
||||
"sort": 0,
|
||||
"value_type": "individual"
|
||||
},
|
||||
"type": "graph",
|
||||
"xaxis": {
|
||||
"buckets": null,
|
||||
"mode": "time",
|
||||
"name": null,
|
||||
"show": true,
|
||||
"values": []
|
||||
},
|
||||
"yaxes": [
|
||||
{
|
||||
"format": "short",
|
||||
"label": null,
|
||||
"logBase": 1,
|
||||
"max": null,
|
||||
"min": null,
|
||||
"show": true
|
||||
},
|
||||
{
|
||||
"format": "short",
|
||||
"label": null,
|
||||
"logBase": 1,
|
||||
"max": null,
|
||||
"min": null,
|
||||
"show": true
|
||||
}
|
||||
],
|
||||
"yaxis": {
|
||||
"align": false,
|
||||
"alignLevel": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"aliasColors": {},
|
||||
"bars": false,
|
||||
"dashLength": 10,
|
||||
"dashes": false,
|
||||
"datasource": "InfluxDB",
|
||||
"fieldConfig": {
|
||||
"defaults": {},
|
||||
"overrides": []
|
||||
},
|
||||
"fill": 1,
|
||||
"fillGradient": 0,
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 10
|
||||
},
|
||||
"hiddenSeries": false,
|
||||
"id": 10,
|
||||
"interval": "1s",
|
||||
"legend": {
|
||||
"avg": false,
|
||||
"current": false,
|
||||
"max": false,
|
||||
"min": false,
|
||||
"show": true,
|
||||
"total": false,
|
||||
"values": false
|
||||
},
|
||||
"lines": true,
|
||||
"linewidth": 1,
|
||||
"maxDataPoints": null,
|
||||
"nullPointMode": "null",
|
||||
"options": {
|
||||
"alertThreshold": true
|
||||
},
|
||||
"percentage": false,
|
||||
"pluginVersion": "7.5.7",
|
||||
"pointradius": 2,
|
||||
"points": false,
|
||||
"renderer": "flot",
|
||||
"seriesOverrides": [],
|
||||
"spaceLength": 10,
|
||||
"stack": false,
|
||||
"steppedLine": false,
|
||||
"targets": [
|
||||
{
|
||||
"groupBy": [
|
||||
{
|
||||
"params": [
|
||||
"$__interval"
|
||||
],
|
||||
"type": "time"
|
||||
},
|
||||
{
|
||||
"params": [
|
||||
"null"
|
||||
],
|
||||
"type": "fill"
|
||||
}
|
||||
],
|
||||
"orderByTime": "ASC",
|
||||
"policy": "default",
|
||||
"query": "from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"tardis_pipeline\")\n |> filter(fn: (r) => r._field == \"utilisation\" or r._field == \"allocation\")\n |> keep(columns: [\"_time\", \"_measurement\", \"_field\", \"_value\", \"tardis_machine_name\"])\n |> aggregateWindow(every: $__interval, fn: mean)\n |> yield()",
|
||||
"queryType": "randomWalk",
|
||||
"refId": "A",
|
||||
"resultFormat": "time_series",
|
||||
"select": [
|
||||
[
|
||||
{
|
||||
"params": [
|
||||
"value"
|
||||
],
|
||||
"type": "field"
|
||||
},
|
||||
{
|
||||
"params": [],
|
||||
"type": "mean"
|
||||
}
|
||||
]
|
||||
],
|
||||
"tags": []
|
||||
}
|
||||
],
|
||||
"thresholds": [],
|
||||
"timeFrom": null,
|
||||
"timeRegions": [],
|
||||
"timeShift": null,
|
||||
"title": "pipeline (utilization/allocation)",
|
||||
"tooltip": {
|
||||
"shared": true,
|
||||
"sort": 0,
|
||||
"value_type": "individual"
|
||||
},
|
||||
"type": "graph",
|
||||
"xaxis": {
|
||||
"buckets": null,
|
||||
"mode": "time",
|
||||
"name": null,
|
||||
"show": true,
|
||||
"values": []
|
||||
},
|
||||
"yaxes": [
|
||||
{
|
||||
"format": "short",
|
||||
"label": null,
|
||||
"logBase": 1,
|
||||
"max": null,
|
||||
"min": null,
|
||||
"show": true
|
||||
},
|
||||
{
|
||||
"format": "short",
|
||||
"label": null,
|
||||
"logBase": 1,
|
||||
"max": null,
|
||||
"min": null,
|
||||
"show": true
|
||||
}
|
||||
],
|
||||
"yaxis": {
|
||||
"align": false,
|
||||
"alignLevel": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"refresh": "5s",
|
||||
"schemaVersion": 27,
|
||||
"style": "dark",
|
||||
"tags": [],
|
||||
"templating": {
|
||||
"list": [
|
||||
{
|
||||
"allValue": null,
|
||||
"current": {
|
||||
"selected": false,
|
||||
"text": "cobald-xvmcqc",
|
||||
"value": "cobald-xvmcqc"
|
||||
},
|
||||
"datasource": "InfluxDB",
|
||||
"definition": "from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> keep(columns: [\"tardis_machine_name\", \"_time\"])\n |> sort(columns: [\"_time\"], desc: true)\n |> unique(column: \"tardis_machine_name\")",
|
||||
"description": null,
|
||||
"error": null,
|
||||
"hide": 0,
|
||||
"includeAll": false,
|
||||
"label": null,
|
||||
"multi": false,
|
||||
"name": "machine",
|
||||
"options": [],
|
||||
"query": "from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> keep(columns: [\"tardis_machine_name\", \"_time\"])\n |> sort(columns: [\"_time\"], desc: true)\n |> unique(column: \"tardis_machine_name\")",
|
||||
"refresh": 2,
|
||||
"regex": "",
|
||||
"skipUrlSync": false,
|
||||
"sort": 0,
|
||||
"tagValuesQuery": "",
|
||||
"tags": [],
|
||||
"tagsQuery": "",
|
||||
"type": "query",
|
||||
"useTags": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"time": {
|
||||
"from": "now-5m",
|
||||
"to": "now"
|
||||
},
|
||||
"timepicker": {},
|
||||
"timezone": "",
|
||||
"title": "cobald",
|
||||
"uid": "urDuvE6Gk",
|
||||
"version": 2
|
||||
}
|
||||
@@ -1,794 +0,0 @@
|
||||
{
|
||||
"meta": {
|
||||
"version": "1",
|
||||
"type": "dashboard",
|
||||
"name": "cobald-Template",
|
||||
"description": "template created from dashboard: cobald"
|
||||
},
|
||||
"content": {
|
||||
"data": {
|
||||
"type": "dashboard",
|
||||
"attributes": {
|
||||
"name": "cobald",
|
||||
"description": ""
|
||||
},
|
||||
"relationships": {
|
||||
"label": {
|
||||
"data": []
|
||||
},
|
||||
"cell": {
|
||||
"data": [
|
||||
{
|
||||
"type": "cell",
|
||||
"id": "07900a722c363000"
|
||||
},
|
||||
{
|
||||
"type": "cell",
|
||||
"id": "07900a7236f63000"
|
||||
},
|
||||
{
|
||||
"type": "cell",
|
||||
"id": "07900a723cf63000"
|
||||
},
|
||||
{
|
||||
"type": "cell",
|
||||
"id": "07900a7243f63000"
|
||||
},
|
||||
{
|
||||
"type": "cell",
|
||||
"id": "079e694f29581000"
|
||||
},
|
||||
{
|
||||
"type": "cell",
|
||||
"id": "079e6e037c181000"
|
||||
}
|
||||
]
|
||||
},
|
||||
"variable": {
|
||||
"data": []
|
||||
}
|
||||
}
|
||||
},
|
||||
"included": [
|
||||
{
|
||||
"id": "07900a722c363000",
|
||||
"type": "cell",
|
||||
"attributes": {
|
||||
"x": 4,
|
||||
"y": 0,
|
||||
"w": 4,
|
||||
"h": 4
|
||||
},
|
||||
"relationships": {
|
||||
"view": {
|
||||
"data": {
|
||||
"type": "view",
|
||||
"id": "07900a722c363000"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "07900a7236f63000",
|
||||
"type": "cell",
|
||||
"attributes": {
|
||||
"x": 0,
|
||||
"y": 0,
|
||||
"w": 4,
|
||||
"h": 4
|
||||
},
|
||||
"relationships": {
|
||||
"view": {
|
||||
"data": {
|
||||
"type": "view",
|
||||
"id": "07900a7236f63000"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "07900a723cf63000",
|
||||
"type": "cell",
|
||||
"attributes": {
|
||||
"x": 4,
|
||||
"y": 4,
|
||||
"w": 4,
|
||||
"h": 4
|
||||
},
|
||||
"relationships": {
|
||||
"view": {
|
||||
"data": {
|
||||
"type": "view",
|
||||
"id": "07900a723cf63000"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "07900a7243f63000",
|
||||
"type": "cell",
|
||||
"attributes": {
|
||||
"x": 0,
|
||||
"y": 4,
|
||||
"w": 4,
|
||||
"h": 4
|
||||
},
|
||||
"relationships": {
|
||||
"view": {
|
||||
"data": {
|
||||
"type": "view",
|
||||
"id": "07900a7243f63000"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "079e694f29581000",
|
||||
"type": "cell",
|
||||
"attributes": {
|
||||
"x": 8,
|
||||
"y": 0,
|
||||
"w": 4,
|
||||
"h": 4
|
||||
},
|
||||
"relationships": {
|
||||
"view": {
|
||||
"data": {
|
||||
"type": "view",
|
||||
"id": "079e694f29581000"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "079e6e037c181000",
|
||||
"type": "cell",
|
||||
"attributes": {
|
||||
"x": 8,
|
||||
"y": 4,
|
||||
"w": 4,
|
||||
"h": 4
|
||||
},
|
||||
"relationships": {
|
||||
"view": {
|
||||
"data": {
|
||||
"type": "view",
|
||||
"id": "079e6e037c181000"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "view",
|
||||
"id": "07900a722c363000",
|
||||
"attributes": {
|
||||
"name": "nodes running",
|
||||
"properties": {
|
||||
"shape": "chronograf-v2",
|
||||
"queries": [
|
||||
{
|
||||
"text": "from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._field == \"state\")\n |> group()\n |> window(every: 10s)\n |> reduce(fn: (r, accumulator) => ({\n _value: accumulator._value + (\n if r._value == \"AvailableState\" then 1 \n else if r._value == \"DownState\" then -1 \n else 0)\n }), identity: {_value: 0})\n |> duplicate(column: \"_stop\", as: \"_time\")\n |> window(every: inf) //, timeColumn: \"_time\")\n |> cumulativeSum()\n// |> reduce(fn: (r, accumulator) => ({r with x: r._value * 2}), identity: {x:0})\n// |> map(fn: (r) => ({r with vnew: r._value*2}))\n// |> integral(unit: 10s, timeColumn: \"_stop\")\n// |> window(every: inf) //, timeColumn: \"_stop\")\n |> yield()",
|
||||
"editMode": "advanced",
|
||||
"name": "",
|
||||
"builderConfig": {
|
||||
"buckets": [],
|
||||
"tags": [
|
||||
{
|
||||
"key": "_measurement",
|
||||
"values": [],
|
||||
"aggregateFunctionType": "filter"
|
||||
}
|
||||
],
|
||||
"functions": [
|
||||
{
|
||||
"name": "mean"
|
||||
}
|
||||
],
|
||||
"aggregateWindow": {
|
||||
"period": "auto",
|
||||
"fillValues": false
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"axes": {
|
||||
"x": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
},
|
||||
"y": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
}
|
||||
},
|
||||
"type": "xy",
|
||||
"legend": {},
|
||||
"geom": "line",
|
||||
"colors": [
|
||||
{
|
||||
"id": "9b960932-18d9-4f57-80ba-24998a06613d",
|
||||
"type": "scale",
|
||||
"hex": "#31C0F6",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "c1742651-0d5e-4148-b9c0-92beb642417a",
|
||||
"type": "scale",
|
||||
"hex": "#A500A5",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "756fa8cb-9d9c-4e45-9a4f-f2b106b0216a",
|
||||
"type": "scale",
|
||||
"hex": "#FF7E27",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
}
|
||||
],
|
||||
"note": "",
|
||||
"showNoteWhenEmpty": false,
|
||||
"xColumn": "_time",
|
||||
"generateXAxisTicks": [],
|
||||
"xTotalTicks": 0,
|
||||
"xTickStart": 0,
|
||||
"xTickStep": 0,
|
||||
"yColumn": "_value",
|
||||
"generateYAxisTicks": [],
|
||||
"yTotalTicks": 0,
|
||||
"yTickStart": 0,
|
||||
"yTickStep": 0,
|
||||
"shadeBelow": false,
|
||||
"position": "overlaid",
|
||||
"timeFormat": "",
|
||||
"hoverDimension": "auto",
|
||||
"legendColorizeRows": true,
|
||||
"legendOpacity": 1,
|
||||
"legendOrientationThreshold": 100000000
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "view",
|
||||
"id": "07900a7236f63000",
|
||||
"attributes": {
|
||||
"name": "Name this Cell",
|
||||
"properties": {
|
||||
"shape": "chronograf-v2",
|
||||
"queries": [
|
||||
{
|
||||
"text": "from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r[\"tardis_machine_name\"] == \"678162c190d5\")\n |> window(every: 10s)\n |> count()\n |> duplicate(column: \"_stop\", as: \"_time\")\n |> window(every: inf)\n |> yield()",
|
||||
"editMode": "advanced",
|
||||
"name": "",
|
||||
"builderConfig": {
|
||||
"buckets": [],
|
||||
"tags": [
|
||||
{
|
||||
"key": "_measurement",
|
||||
"values": [],
|
||||
"aggregateFunctionType": "filter"
|
||||
}
|
||||
],
|
||||
"functions": [
|
||||
{
|
||||
"name": "mean"
|
||||
}
|
||||
],
|
||||
"aggregateWindow": {
|
||||
"period": "auto",
|
||||
"fillValues": false
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"axes": {
|
||||
"x": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
},
|
||||
"y": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
}
|
||||
},
|
||||
"type": "xy",
|
||||
"legend": {},
|
||||
"geom": "line",
|
||||
"colors": [
|
||||
{
|
||||
"id": "2566435b-7ee0-4222-8ac0-b7f14ab783d9",
|
||||
"type": "scale",
|
||||
"hex": "#31C0F6",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "9263bcff-35a0-4025-bacd-68a1bef54784",
|
||||
"type": "scale",
|
||||
"hex": "#A500A5",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "2f04bd8c-2203-4be6-bc34-c25720d24379",
|
||||
"type": "scale",
|
||||
"hex": "#FF7E27",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
}
|
||||
],
|
||||
"note": "",
|
||||
"showNoteWhenEmpty": false,
|
||||
"xColumn": "_time",
|
||||
"generateXAxisTicks": [],
|
||||
"xTotalTicks": 0,
|
||||
"xTickStart": 0,
|
||||
"xTickStep": 0,
|
||||
"yColumn": "_value",
|
||||
"generateYAxisTicks": [],
|
||||
"yTotalTicks": 0,
|
||||
"yTickStart": 0,
|
||||
"yTickStep": 0,
|
||||
"shadeBelow": false,
|
||||
"position": "overlaid",
|
||||
"timeFormat": "",
|
||||
"hoverDimension": "auto",
|
||||
"legendColorizeRows": true,
|
||||
"legendOpacity": 1,
|
||||
"legendOrientationThreshold": 100000000
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "view",
|
||||
"id": "07900a723cf63000",
|
||||
"attributes": {
|
||||
"name": "node fluctuation",
|
||||
"properties": {
|
||||
"shape": "chronograf-v2",
|
||||
"queries": [
|
||||
{
|
||||
"text": "f = (r, accumulator) => ({\n _value: accumulator._value + (if r._value == \"AvailableState\" then 1 else if r._value == \"DownState\" then -1 else 0)\n })\n\nfrom(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._field == \"state\")\n |> group()\n |> window(every: 10s)\n |> reduce(fn: f, identity: {_value: 0})\n |> duplicate(column: \"_stop\", as: \"_time\")\n |> window(every: inf, timeColumn: \"_time\")\n |> yield()",
|
||||
"editMode": "advanced",
|
||||
"name": "",
|
||||
"builderConfig": {
|
||||
"buckets": [],
|
||||
"tags": [
|
||||
{
|
||||
"key": "_measurement",
|
||||
"values": [],
|
||||
"aggregateFunctionType": "filter"
|
||||
}
|
||||
],
|
||||
"functions": [
|
||||
{
|
||||
"name": "mean"
|
||||
}
|
||||
],
|
||||
"aggregateWindow": {
|
||||
"period": "auto",
|
||||
"fillValues": false
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"axes": {
|
||||
"x": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
},
|
||||
"y": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
}
|
||||
},
|
||||
"type": "xy",
|
||||
"legend": {},
|
||||
"geom": "line",
|
||||
"colors": [
|
||||
{
|
||||
"id": "9b960932-18d9-4f57-80ba-24998a06613d",
|
||||
"type": "scale",
|
||||
"hex": "#31C0F6",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "c1742651-0d5e-4148-b9c0-92beb642417a",
|
||||
"type": "scale",
|
||||
"hex": "#A500A5",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "756fa8cb-9d9c-4e45-9a4f-f2b106b0216a",
|
||||
"type": "scale",
|
||||
"hex": "#FF7E27",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
}
|
||||
],
|
||||
"note": "",
|
||||
"showNoteWhenEmpty": false,
|
||||
"xColumn": "_time",
|
||||
"generateXAxisTicks": [],
|
||||
"xTotalTicks": 0,
|
||||
"xTickStart": 0,
|
||||
"xTickStep": 0,
|
||||
"yColumn": "_value",
|
||||
"generateYAxisTicks": [],
|
||||
"yTotalTicks": 0,
|
||||
"yTickStart": 0,
|
||||
"yTickStep": 0,
|
||||
"shadeBelow": false,
|
||||
"position": "overlaid",
|
||||
"timeFormat": "",
|
||||
"hoverDimension": "auto",
|
||||
"legendColorizeRows": true,
|
||||
"legendOpacity": 1,
|
||||
"legendOrientationThreshold": 100000000
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "view",
|
||||
"id": "07900a7243f63000",
|
||||
"attributes": {
|
||||
"name": "states",
|
||||
"properties": {
|
||||
"shape": "chronograf-v2",
|
||||
"queries": [
|
||||
{
|
||||
"text": "from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._field == \"state\")\n |> group(columns: [\"_value\"], mode: \"by\")\n |> duplicate(column: \"_value\", as: \"state\")\n |> window(every: 10s)\n |> count(column: \"state\")\n |> rename(columns: {\"_value\": \"_field\", \"state\": \"_value\"})\n |> group(columns: [\"_field\"])\n |> yield()",
|
||||
"editMode": "advanced",
|
||||
"name": "",
|
||||
"builderConfig": {
|
||||
"buckets": [],
|
||||
"tags": [
|
||||
{
|
||||
"key": "_measurement",
|
||||
"values": [],
|
||||
"aggregateFunctionType": "filter"
|
||||
}
|
||||
],
|
||||
"functions": [
|
||||
{
|
||||
"name": "mean"
|
||||
}
|
||||
],
|
||||
"aggregateWindow": {
|
||||
"period": "auto",
|
||||
"fillValues": false
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"axes": {
|
||||
"x": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
},
|
||||
"y": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
}
|
||||
},
|
||||
"type": "xy",
|
||||
"legend": {},
|
||||
"geom": "line",
|
||||
"colors": [
|
||||
{
|
||||
"id": "9b960932-18d9-4f57-80ba-24998a06613d",
|
||||
"type": "scale",
|
||||
"hex": "#31C0F6",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "c1742651-0d5e-4148-b9c0-92beb642417a",
|
||||
"type": "scale",
|
||||
"hex": "#A500A5",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "756fa8cb-9d9c-4e45-9a4f-f2b106b0216a",
|
||||
"type": "scale",
|
||||
"hex": "#FF7E27",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
}
|
||||
],
|
||||
"note": "",
|
||||
"showNoteWhenEmpty": false,
|
||||
"xColumn": "_stop",
|
||||
"generateXAxisTicks": [],
|
||||
"xTotalTicks": 0,
|
||||
"xTickStart": 0,
|
||||
"xTickStep": 0,
|
||||
"yColumn": "_value",
|
||||
"generateYAxisTicks": [],
|
||||
"yTotalTicks": 0,
|
||||
"yTickStart": 0,
|
||||
"yTickStep": 0,
|
||||
"shadeBelow": false,
|
||||
"position": "overlaid",
|
||||
"timeFormat": "",
|
||||
"hoverDimension": "auto",
|
||||
"legendColorizeRows": true,
|
||||
"legendOpacity": 1,
|
||||
"legendOrientationThreshold": 100000000
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "view",
|
||||
"id": "079e694f29581000",
|
||||
"attributes": {
|
||||
"name": "pipeline (demand / supply)",
|
||||
"properties": {
|
||||
"shape": "chronograf-v2",
|
||||
"queries": [
|
||||
{
|
||||
"text": "from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"tardis_pipeline\")\n |> filter(fn: (r) => r._field == \"demand\" or r._field == \"supply\")\n// |> filter(fn: (r) => r._field == \"state\")\n// |> group()\n// |> window(every: 10s)\n// |> duplicate(column: \"_stop\", as: \"_time\")\n// |> window(every: inf) //, timeColumn: \"_time\")\n// |> cumulativeSum()\n |> yield()",
|
||||
"editMode": "advanced",
|
||||
"name": "",
|
||||
"builderConfig": {
|
||||
"buckets": [],
|
||||
"tags": [
|
||||
{
|
||||
"key": "_measurement",
|
||||
"values": [],
|
||||
"aggregateFunctionType": "filter"
|
||||
}
|
||||
],
|
||||
"functions": [
|
||||
{
|
||||
"name": "mean"
|
||||
}
|
||||
],
|
||||
"aggregateWindow": {
|
||||
"period": "auto",
|
||||
"fillValues": false
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"axes": {
|
||||
"x": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
},
|
||||
"y": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
}
|
||||
},
|
||||
"type": "xy",
|
||||
"legend": {},
|
||||
"geom": "line",
|
||||
"colors": [
|
||||
{
|
||||
"id": "4ef29481-ecf3-4a09-b0f5-e34e8d3e50b5",
|
||||
"type": "scale",
|
||||
"hex": "#31C0F6",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "719de04a-f70f-4c54-a1bb-982a9d13dbae",
|
||||
"type": "scale",
|
||||
"hex": "#A500A5",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "8d5c1f25-3801-4cdd-ad40-b8e2e78342a0",
|
||||
"type": "scale",
|
||||
"hex": "#FF7E27",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
}
|
||||
],
|
||||
"note": "",
|
||||
"showNoteWhenEmpty": false,
|
||||
"xColumn": "_time",
|
||||
"generateXAxisTicks": [],
|
||||
"xTotalTicks": 0,
|
||||
"xTickStart": 0,
|
||||
"xTickStep": 0,
|
||||
"yColumn": "_value",
|
||||
"generateYAxisTicks": [],
|
||||
"yTotalTicks": 0,
|
||||
"yTickStart": 0,
|
||||
"yTickStep": 0,
|
||||
"shadeBelow": false,
|
||||
"position": "overlaid",
|
||||
"timeFormat": "",
|
||||
"hoverDimension": "auto",
|
||||
"legendColorizeRows": true,
|
||||
"legendOpacity": 1,
|
||||
"legendOrientationThreshold": 100000000
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "view",
|
||||
"id": "079e6e037c181000",
|
||||
"attributes": {
|
||||
"name": "pipeline (Utilization / Allocation)",
|
||||
"properties": {
|
||||
"shape": "chronograf-v2",
|
||||
"queries": [
|
||||
{
|
||||
"text": "from(bucket: \"batleth\")\n |> range(start: v.timeRangeStart, stop: v.timeRangeStop)\n |> filter(fn: (r) => r._measurement == \"tardis_pipeline\")\n |> filter(fn: (r) => r._field == \"utilisation\" or r._field == \"allocation\")\n// |> filter(fn: (r) => r._field == \"state\")\n// |> group()\n// |> window(every: 10s)\n// |> duplicate(column: \"_stop\", as: \"_time\")\n// |> window(every: inf) //, timeColumn: \"_time\")\n// |> cumulativeSum()\n |> yield()",
|
||||
"editMode": "advanced",
|
||||
"name": "",
|
||||
"builderConfig": {
|
||||
"buckets": [],
|
||||
"tags": [
|
||||
{
|
||||
"key": "_measurement",
|
||||
"values": [],
|
||||
"aggregateFunctionType": "filter"
|
||||
}
|
||||
],
|
||||
"functions": [
|
||||
{
|
||||
"name": "mean"
|
||||
}
|
||||
],
|
||||
"aggregateWindow": {
|
||||
"period": "auto",
|
||||
"fillValues": false
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"axes": {
|
||||
"x": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
},
|
||||
"y": {
|
||||
"bounds": [
|
||||
"",
|
||||
""
|
||||
],
|
||||
"label": "",
|
||||
"prefix": "",
|
||||
"suffix": "",
|
||||
"base": "10",
|
||||
"scale": "linear"
|
||||
}
|
||||
},
|
||||
"type": "xy",
|
||||
"legend": {},
|
||||
"geom": "line",
|
||||
"colors": [
|
||||
{
|
||||
"id": "4ef29481-ecf3-4a09-b0f5-e34e8d3e50b5",
|
||||
"type": "scale",
|
||||
"hex": "#31C0F6",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "719de04a-f70f-4c54-a1bb-982a9d13dbae",
|
||||
"type": "scale",
|
||||
"hex": "#A500A5",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"id": "8d5c1f25-3801-4cdd-ad40-b8e2e78342a0",
|
||||
"type": "scale",
|
||||
"hex": "#FF7E27",
|
||||
"name": "Nineteen Eighty Four",
|
||||
"value": 0
|
||||
}
|
||||
],
|
||||
"note": "",
|
||||
"showNoteWhenEmpty": false,
|
||||
"xColumn": "_time",
|
||||
"generateXAxisTicks": [],
|
||||
"xTotalTicks": 0,
|
||||
"xTickStart": 0,
|
||||
"xTickStep": 0,
|
||||
"yColumn": "_value",
|
||||
"generateYAxisTicks": [],
|
||||
"yTotalTicks": 0,
|
||||
"yTickStart": 0,
|
||||
"yTickStep": 0,
|
||||
"shadeBelow": false,
|
||||
"position": "overlaid",
|
||||
"timeFormat": "",
|
||||
"hoverDimension": "auto",
|
||||
"legendColorizeRows": true,
|
||||
"legendOpacity": 1,
|
||||
"legendOrientationThreshold": 100000000
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"labels": []
|
||||
}
|
||||
@@ -1,6 +0,0 @@
|
||||
[influxdb]
|
||||
name = InfluxDB Repository - RHEL \$releasever
|
||||
baseurl = https://repos.influxdata.com/rhel/\$releasever/\$basearch/stable
|
||||
enabled = 1
|
||||
gpgcheck = 1
|
||||
gpgkey = https://repos.influxdata.com/influxdb.key
|
||||
@@ -1,5 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
[ -f /usr/local/lib/cobaldmodules/setup.py -a \
|
||||
-d /usr/local/lib/cobaldmodules/cobaldmodules ] && \
|
||||
pip3 install --no-deps --editable /usr/local/lib/cobaldmodules
|
||||
@@ -1,31 +0,0 @@
|
||||
Bootstrap: docker-daemon
|
||||
From: slurm:slurmd
|
||||
|
||||
%files
|
||||
31-slurmd-configless /etc/docker-init.d/31-slurm-configless
|
||||
/container/volumes/munge/munge.key /etc/munge/munge.key
|
||||
cgroup.conf.noautomount /etc/slurm/cgroup.conf
|
||||
|
||||
%post
|
||||
rm /etc/docker-init.d/30-slurmd
|
||||
chmod 755 /etc/docker-init.d/31-slurm-configless
|
||||
|
||||
%startscript
|
||||
if [ -z "${1}" -o -z "${2}" ] ; then
|
||||
echo "undefined variables slurmctld or nodename"
|
||||
exit 1
|
||||
fi
|
||||
export slurmctld="${1}"
|
||||
export nodename="${2}"
|
||||
echo ${slurmctld} ${nodename} ${SLURMD_LOG_PATH} ${SLURM_SCHED_LOG_PATH}
|
||||
exec /usr/local/sbin/entrypoint.sh /usr/local/sbin/docker-init
|
||||
|
||||
%runscript
|
||||
if [ -z "${1}" -o -z "${2}" ] ; then
|
||||
echo "undefined variables slurmctld or nodename"
|
||||
exit 1
|
||||
fi
|
||||
export slurmctld="${1}"
|
||||
export nodename="${2}"
|
||||
echo ${slurmctld} ${nodename} ${SLURMD_LOG_PATH} ${SLURM_SCHED_LOG_PATH}
|
||||
exec /usr/local/sbin/entrypoint.sh /usr/local/sbin/docker-init
|
||||
@@ -1,59 +0,0 @@
|
||||
#!/bin/sh
|
||||
#SBATCH -D /shared
|
||||
export
|
||||
echo $@
|
||||
nodename=$(hostname | awk '{ print "drone" substr($1,match($1, "([[:digit:]]+)")) }')
|
||||
|
||||
SHUTDOWN_DONE=0
|
||||
|
||||
function handler_quit(){
|
||||
[ $SHUTDOWN_DONE -ne 0 ] && return
|
||||
set -x
|
||||
echo "drain container"
|
||||
scontrol update NodeName=${nodename} State=DRAIN Reason="cobald node quit"
|
||||
shutdown_jobs=$(squeue -w ${nodename} --noheader -O jobid)
|
||||
[ -n "${shutdown_jobs}" ] && scancel ${shutdown_jobs}
|
||||
#scancel -w ${nodename}
|
||||
i=$(( $(scontrol show config | grep KillWait | \
|
||||
sed 's/^KillWait.*= \([0-9]*\) sec/\1/') - 2 ))
|
||||
while [ -n "$(squeue -w ${nodename} --noheader -O jobid)" -o ${i} -lt 1 ]
|
||||
do
|
||||
i=$(( ${i} - 1 ))
|
||||
sleep 1
|
||||
done
|
||||
scancel -s KILL -w ${nodename} # hard kill all remaining jobs
|
||||
echo "shutdown container"
|
||||
scontrol update NodeName=${nodename} State=DOWN Reason=shutdown
|
||||
singularity instance stop slurm-drone
|
||||
scontrol update NodeName=${nodename} State=FUTURE
|
||||
umount /inner-cgroup/freezer
|
||||
umount /inner-cgroup
|
||||
SHUTDOWN_DONE=1
|
||||
exit 0
|
||||
}
|
||||
|
||||
# set -x
|
||||
|
||||
trap handler_quit EXIT
|
||||
|
||||
echo "mounting cgroups"
|
||||
mkdir /inner-cgroup
|
||||
mount -t tmpfs none /inner-cgroup
|
||||
mkdir /inner-cgroup/freezer/
|
||||
mount --bind /sys/fs/cgroup/freezer/slurm/ /inner-cgroup/freezer/
|
||||
mount -o remount,ro /inner-cgroup
|
||||
|
||||
echo "starting ${nodename}"
|
||||
scontrol update NodeName=${nodename} State=RESUME # revoke last DRAIN
|
||||
scontrol update NodeName=${nodename} State=FUTURE
|
||||
singularity instance start \
|
||||
-B /inner-cgroup/:/sys/fs/cgroup/ \
|
||||
--writable-tmpfs /shared/slurmd.sif slurm-drone \
|
||||
slurm-ctl ${nodename}
|
||||
# scontrol update NodeName=${nodename} NodeHostname=${SLURM_JOB_ID}
|
||||
scontrol update NodeName=${nodename} NodeHostname=${TardisDroneUuid}
|
||||
if [ $? -eq 0 ] ; then
|
||||
echo "container started, sleeping $(( 60 * ${SLURM_Walltime} - 2 ))"
|
||||
sleep $(( 60 * ${SLURM_Walltime} - 2 ))
|
||||
fi
|
||||
handler_quit
|
||||
@@ -1,8 +0,0 @@
|
||||
FROM centos:7
|
||||
|
||||
COPY influxdb.repo /etc/yum.repos.d/influxdb.repo
|
||||
|
||||
RUN yum -y install telegraf &&\
|
||||
yum clean all && rm -rf /var/cache/yum
|
||||
|
||||
CMD telegraf
|
||||
@@ -1,11 +0,0 @@
|
||||
# Tests
|
||||
ANSIBLE_LIBRARY=. ansible -m influx_bucket -a "base='http://192.168.122.140:28086' org='my-org' auth_token='87-fEnSlQldFi1T_CLHsrHxH-T9VKey-qzUbVH6tmR2QzL4oZzbUPwzS1wzOoIkyfmyGbRv75yLjYfztxziivw==' name='bucky' description='test 123'" localhost -vvv
|
||||
ANSIBLE_LIBRARY=. ansible -m influx_token -a "base='http://192.168.122.140:28086' org='my-org' auth_token='87-fEnSlQldFi1T_CLHsrHxH-T9VKey-qzUbVH6tmR2QzL4oZzbUPwzS1wzOoIkyfmyGbRv75yLjYfztxziivw==' key='foo' description='test 123' permissions=\"{{'[{\\\"action\\\": \\\"write\\\",\\\"resource\\\": {\\\"type\\\": \\\"buckets\\\"} }]'|from_json}}\"" localhost -vvv
|
||||
ANSIBLE_LIBRARY=. ansible -m influx_dashboard -create -a "base='http://192.168.122.140:28086' org='my-org' token='2Mji-PvTzgn2oie5p36pJ-vxqWCnxczMWGrnYz2nUHj6Q6XvdIGiLPmK4DjX16KGhOjxQ5dWymDusE8qJrhFFg==' data='{{lookup(\"file\", \"../files/influxdb-dashboard-cobald.json\")}}'" localhost -vvv
|
||||
Missing: lot of stuff, e.g. missing tokens, invalid data like bucket instead of buckets ...
|
||||
|
||||
# TODO
|
||||
* tests
|
||||
* state (present/absent)
|
||||
* `module_utils/urls.py` (https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/urls.py)
|
||||
* see module notes
|
||||
@@ -1,161 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
import requests
|
||||
from ansible.module_utils.basic import AnsibleModule
|
||||
|
||||
DOCUMENTATION = r'''
|
||||
---
|
||||
module: influx2_bucket
|
||||
short_description: create bucket in influxdb2
|
||||
description: create bucket in influxdb2
|
||||
notes:
|
||||
- just works with influxdb version 2
|
||||
- does not remove buckets
|
||||
- no way to configure data retention
|
||||
|
||||
options:
|
||||
base:
|
||||
description: URL for path, e.g. `https://localhost:8086`
|
||||
type: str
|
||||
required: True
|
||||
org:
|
||||
description: influxdb2 organisation
|
||||
type: str
|
||||
required: True
|
||||
auth_token:
|
||||
description: influxdb2 authentication token
|
||||
type: str
|
||||
required: True
|
||||
name:
|
||||
description: name of the bucket
|
||||
type: str
|
||||
required: True
|
||||
force:
|
||||
description: force creation even if bucket already exists
|
||||
(adds a new one)
|
||||
type: bool
|
||||
required: False
|
||||
default: False
|
||||
author:
|
||||
- Thorsten M. (@thoto)
|
||||
'''
|
||||
|
||||
EXAMPLES = r'''
|
||||
- name: "fetch auth token"
|
||||
raw: influx auth list --user my-user --hide-headers | cut -f 3
|
||||
register: influx_token_fetch
|
||||
delegate_to: ed-influxdb-2
|
||||
|
||||
- name: "create bucket"
|
||||
influx_bucket:
|
||||
base: "http://localhost:8086"
|
||||
org: "my-org"
|
||||
token: "{{influx_token_fetch.stdout_lines[0]}}"
|
||||
name: "bucky"
|
||||
'''
|
||||
|
||||
|
||||
def get_org_id(base, org_name, h):
|
||||
ro = requests.get("{base}/api/v2/orgs".format(base=base), headers=h,
|
||||
json={"org": org_name})
|
||||
ro.raise_for_status()
|
||||
org_id = [o["id"] for o in ro.json()["orgs"] if o["name"] == org_name]
|
||||
return org_id[0]
|
||||
|
||||
|
||||
class Bucket:
|
||||
def __init__(self, base, h, org, description, name):
|
||||
self.base = base
|
||||
self.h = h
|
||||
self.org_id = get_org_id(base, org, h)
|
||||
self.description = description
|
||||
self.name = name
|
||||
|
||||
self.result = None
|
||||
self.f = None
|
||||
|
||||
def check(self):
|
||||
ra = requests.get("{base}/api/v2/buckets".format(
|
||||
base=self.base),
|
||||
params={"orgID": self.org_id, "name": self.name},
|
||||
headers=self.h)
|
||||
if ra.status_code == 404:
|
||||
# orgID + name -> 404 on empty set. Just name -> 200 but buckets=[]
|
||||
x = []
|
||||
else:
|
||||
ra.raise_for_status()
|
||||
x = [i for i in ra.json()["buckets"]
|
||||
if self.name == i["name"] and i["orgID"] == self.org_id]
|
||||
assert(len(x) == 1 or len(x) == 0)
|
||||
|
||||
update = None
|
||||
if len(x) == 0: # create
|
||||
self.result = None
|
||||
self.f = lambda: self._create({
|
||||
"orgID": self.org_id,
|
||||
"description": self.description if self.description else None,
|
||||
"name": self.name,
|
||||
"retentionRules": []
|
||||
})
|
||||
else:
|
||||
self.result = x[0]
|
||||
if self.description == x[0].get("description", ""):
|
||||
return False # everything matches -> no change needed
|
||||
else:
|
||||
self.result = x[0]
|
||||
update = {"id": x[0]["id"],
|
||||
"description": self.description}
|
||||
self.f = lambda: self._update(**update)
|
||||
return True
|
||||
|
||||
def run(self):
|
||||
if not self.f:
|
||||
self.check()
|
||||
self.f()
|
||||
|
||||
def _update(self, id, description):
|
||||
ra = requests.patch(
|
||||
"{base}/api/v2/buckets/{id}".format(
|
||||
base=self.base, id=id),
|
||||
headers=self.h, json={"description": description})
|
||||
ra.raise_for_status()
|
||||
return ra
|
||||
|
||||
def _create(self, data):
|
||||
ra = requests.post("{base}/api/v2/buckets".format(base=self.base),
|
||||
headers=self.h, json=data)
|
||||
ra.raise_for_status()
|
||||
self.result = ra.json()
|
||||
return ra
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
result = dict(changed=False, message="")
|
||||
module = AnsibleModule(
|
||||
argument_spec=dict(
|
||||
base=dict(type="str", required=True),
|
||||
org=dict(type="str", required=True),
|
||||
auth_token=dict(type="str", required=True),
|
||||
name=dict(type="str", required=True),
|
||||
description=dict(type="str", default=""),
|
||||
force=dict(type="bool", default=False),
|
||||
),
|
||||
supports_check_mode=True
|
||||
)
|
||||
h = {"Authorization": "Token {token}".format(
|
||||
token=module.params["auth_token"])}
|
||||
b = Bucket(module.params["base"], h, org=module.params["org"],
|
||||
description=module.params["description"],
|
||||
name=module.params["name"])
|
||||
|
||||
changed = b.check()
|
||||
if b.result:
|
||||
result['bucket_id'] = b.result["id"]
|
||||
result['changed'] = changed
|
||||
|
||||
if module.check_mode:
|
||||
module.exit_json(**result)
|
||||
|
||||
if changed or module.params["force"]:
|
||||
b.run()
|
||||
result['bucket_id'] = b.result["id"]
|
||||
module.exit_json(**result)
|
||||
@@ -1,147 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
import json
|
||||
import requests
|
||||
from ansible.module_utils.basic import AnsibleModule
|
||||
|
||||
DOCUMENTATION = r'''
|
||||
---
|
||||
module: influx2_dashboard
|
||||
short_description: create dashboard in influxdb2
|
||||
description: create dashboard in influxdb2
|
||||
notes:
|
||||
- just works with influxdb version 2
|
||||
- does not create dashboard description
|
||||
- does not update dashboards
|
||||
- just creates a dashboard if it does not exist.
|
||||
|
||||
options:
|
||||
base:
|
||||
description: URL for path, e.g. `https://localhost:8086`
|
||||
type: str
|
||||
required: True
|
||||
org:
|
||||
description: influxdb2 organisation
|
||||
type: str
|
||||
required: True
|
||||
auth_token:
|
||||
description: influxdb2 authentication token
|
||||
type: str
|
||||
required: True
|
||||
data:
|
||||
description: exported dashboard json file
|
||||
type: json
|
||||
required: True
|
||||
force:
|
||||
description: force creation even if dashboard already exists
|
||||
(adds a new one)
|
||||
type: bool
|
||||
required: False
|
||||
default: False
|
||||
author:
|
||||
- Thorsten M. (@thoto)
|
||||
'''
|
||||
|
||||
EXAMPLES = r'''
|
||||
- name: "fetch auth token"
|
||||
raw: influx auth list --user my-user --hide-headers | cut -f 3
|
||||
register: influx_token_fetch
|
||||
delegate_to: ed-influxdb-2
|
||||
|
||||
- name: "create dashboard"
|
||||
influx_dashboard:
|
||||
base: "http://localhost:8086"
|
||||
org: "my-org"
|
||||
auth_token: "{{influx_token_fetch.stdout_lines[0]}}"
|
||||
data: "{{lookup('file', 'influxdb-dashboard-cobald.json')}}"
|
||||
'''
|
||||
|
||||
|
||||
def parse_dashboard_template(data):
|
||||
j = json.loads(data)
|
||||
|
||||
return {
|
||||
'name': j["content"]["data"]["attributes"]["name"],
|
||||
'cells': {i["relationships"]["view"]["data"]["id"]: i["attributes"]
|
||||
for i in j["content"]["included"] if i["type"] == "cell"},
|
||||
'views': {i["id"]: i["attributes"]
|
||||
for i in j["content"]["included"] if i["type"] == "view"},
|
||||
}
|
||||
|
||||
|
||||
def get_auth(base, org_name, token):
|
||||
h = {"Authorization": "Token {token}".format(token=token)}
|
||||
|
||||
rm = requests.get("{base}/api/v2/me".format(base=base), headers=h)
|
||||
rm.raise_for_status()
|
||||
# me_name = rm.json()["name"]
|
||||
me = rm.json()["id"]
|
||||
|
||||
ro = requests.get("{base}/api/v2/orgs".format(base=base), headers=h,
|
||||
json={"userID": me})
|
||||
ro.raise_for_status()
|
||||
org_id = [o["id"] for o in ro.json()["orgs"] if o["name"] == org_name]
|
||||
|
||||
return {"org_id": org_id[0], "org_name": org_name, "uid": me, "h": h}
|
||||
|
||||
|
||||
def check(base, auth, dashboard):
|
||||
h = auth["h"]
|
||||
rd = requests.get("{base}/api/v2/dashboards".format(base=base), headers=h)
|
||||
rd.raise_for_status()
|
||||
x = [i for i in rd.json()["dashboards"]
|
||||
if i["name"] == dashboard["name"] and i["orgID"] == auth["org_id"]]
|
||||
return len(x) == 0, x
|
||||
|
||||
|
||||
def create(base, auth, dashboard):
|
||||
h = auth["h"]
|
||||
|
||||
# create dashboard
|
||||
rd = requests.post("{base}/api/v2/dashboards".format(base=base), headers=h,
|
||||
json={"orgID": auth["org_id"],
|
||||
"name": dashboard["name"]})
|
||||
rd.raise_for_status()
|
||||
dash_id = rd.json()["id"]
|
||||
|
||||
for k, v in dashboard["cells"].items():
|
||||
# create cells in dashboard
|
||||
rc = requests.post(
|
||||
"{base}/api/v2/dashboards/{dash_id}/cells".format(
|
||||
base=base, dash_id=dash_id),
|
||||
headers=h, json=v)
|
||||
rc.raise_for_status()
|
||||
cell_id = rc.json()["id"]
|
||||
|
||||
# create view of cell in dashboard
|
||||
rv = requests.patch(
|
||||
"{base}/api/v2/dashboards/{dash_id}/cells/{cell_id}/view".format(
|
||||
base=base, dash_id=dash_id, cell_id=cell_id),
|
||||
headers=h, json=dashboard["views"][k])
|
||||
rv.raise_for_status()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
result = dict(changed=False, message="")
|
||||
module = AnsibleModule(
|
||||
argument_spec=dict(
|
||||
base=dict(type="str", required=True),
|
||||
org=dict(type="str", required=True),
|
||||
auth_token=dict(type="str", required=True),
|
||||
data=dict(type="json", required=True),
|
||||
force=dict(type="bool", default=False),
|
||||
),
|
||||
supports_check_mode=True
|
||||
)
|
||||
dashboard = parse_dashboard_template(module.params["data"])
|
||||
auth = get_auth(module.params["base"], module.params["org"],
|
||||
module.params["auth_token"])
|
||||
changed, x = check(module.params["base"], auth, dashboard)
|
||||
|
||||
result['changed'] = changed
|
||||
|
||||
if module.check_mode:
|
||||
module.exit_json(**result)
|
||||
|
||||
if changed or module.params["force"]:
|
||||
create(module.params["base"], auth, dashboard)
|
||||
module.exit_json(**result)
|
||||
@@ -1,232 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
import requests
|
||||
from ansible.module_utils.basic import AnsibleModule
|
||||
|
||||
DOCUMENTATION = r'''
|
||||
---
|
||||
module: influx2_token
|
||||
short_description: generate token via influxdb2 api
|
||||
description: generate token via influxdb2 api
|
||||
notes:
|
||||
- just works with influxdb version 2
|
||||
- needs token to authenticate against API (use
|
||||
`influx auth list --user my-user --hide-headers | cut -f 3`
|
||||
- tokens may not be removed
|
||||
- permissions can not be updated. a new token is created and the old
|
||||
one is not removed.
|
||||
options:
|
||||
base:
|
||||
description: URL for path, e.g. `https://localhost:8086`
|
||||
type: str
|
||||
required: True
|
||||
org:
|
||||
description: influxdb2 organisation
|
||||
type: str
|
||||
required: True
|
||||
auth_token:
|
||||
description: influxdb2 authentication token
|
||||
type: str
|
||||
required: True
|
||||
key:
|
||||
description: some key used to identify token. This is put into
|
||||
the tokens description
|
||||
type: str
|
||||
required: True
|
||||
description:
|
||||
description: textual description for token. key gets appended
|
||||
type: str
|
||||
required: False
|
||||
permissions:
|
||||
description: list of permissions, each dict(action, resource)
|
||||
type: list
|
||||
required: True
|
||||
force:
|
||||
description: force creation even if dashboard already exists
|
||||
(adds a new one)
|
||||
type: bool
|
||||
required: False
|
||||
default: False
|
||||
author:
|
||||
- Thorsten M. (@thoto)
|
||||
'''
|
||||
|
||||
EXAMPLES = r'''
|
||||
- name: "fetch auth token"
|
||||
raw: influx auth list --user my-user --hide-headers | cut -f 3
|
||||
register: influx_token_fetch
|
||||
delegate_to: ed-influxdb-2
|
||||
|
||||
- name: "create dashboard"
|
||||
influx_token:
|
||||
base: "http://localhost:8086"
|
||||
org: "my-org"
|
||||
auth_token: "{{influx_token_fetch.stdout_lines[0]}}"
|
||||
key: "foo123"
|
||||
description: "token for foo key"
|
||||
permissions:
|
||||
- action: "write"
|
||||
resource:
|
||||
type: "buckets"
|
||||
register: ed-influx-token
|
||||
|
||||
- debug: msg="Token: {{ed-influx-token.token}}"
|
||||
'''
|
||||
|
||||
|
||||
def get_org_id(base, org_name, h):
|
||||
ro = requests.get("{base}/api/v2/orgs".format(base=base), headers=h,
|
||||
json={"org": org_name})
|
||||
ro.raise_for_status()
|
||||
org_id = [o["id"] for o in ro.json()["orgs"] if o["name"] == org_name]
|
||||
return org_id[0]
|
||||
|
||||
|
||||
def marker(key):
|
||||
return "__ANSIBLE_TOKEN_{key}__".format(key=key)
|
||||
|
||||
|
||||
def _filter_perms(perms):
|
||||
for r in perms:
|
||||
r["resource"] = {k: v for k, v in r["resource"].items() if v}
|
||||
return perms
|
||||
|
||||
|
||||
class Token:
|
||||
def __init__(self, base, h, data):
|
||||
self.base = base
|
||||
self.h = h
|
||||
self.marker = marker(data["key"])
|
||||
|
||||
self.org_id = data["org_id"]
|
||||
self.perms = _filter_perms(data["perms"])
|
||||
self.description = data["description"]+" "+self.marker
|
||||
|
||||
self.result_token = None
|
||||
self.f = None
|
||||
|
||||
def check(self):
|
||||
ra = requests.get("{base}/api/v2/authorizations".format(
|
||||
base=self.base),
|
||||
params={"orgID": self.org_id},
|
||||
headers=self.h)
|
||||
ra.raise_for_status()
|
||||
|
||||
update = None
|
||||
for i in ra.json()["authorizations"]:
|
||||
if self.marker not in i["description"] \
|
||||
or i["orgID"] != self.org_id:
|
||||
continue
|
||||
if self._match_perms(self.perms, i["permissions"]):
|
||||
self.result_token = i
|
||||
if self.description == i["description"]:
|
||||
return False # everything matches -> no change needed
|
||||
else:
|
||||
update = {"auth_id": i["id"],
|
||||
"description": self.description}
|
||||
# TODO: may remove token because permissions do not match?
|
||||
if update:
|
||||
self.f = lambda: self._update(**update)
|
||||
else:
|
||||
self.result_token = None
|
||||
self.f = lambda: self._create({
|
||||
"orgID": self.org_id,
|
||||
"description": self.description,
|
||||
"permissions": self.perms
|
||||
})
|
||||
return True
|
||||
|
||||
def run(self):
|
||||
if not self.f:
|
||||
self.check()
|
||||
self.f()
|
||||
|
||||
def _match_perms(self, pa, pb):
|
||||
def g(match, lst):
|
||||
for idx, i in enumerate(lst):
|
||||
if i['action'] != match['action']:
|
||||
continue
|
||||
for k, v in match['resource'].items():
|
||||
if k not in i['resource'] or i['resource'][k] != v:
|
||||
continue
|
||||
else: # first best match
|
||||
return idx
|
||||
else:
|
||||
raise ValueError
|
||||
|
||||
b = [b.copy() for b in pb]
|
||||
for i in pa:
|
||||
try:
|
||||
b.pop(g(i, b))
|
||||
except ValueError:
|
||||
return False # permission i not present in b
|
||||
|
||||
if b: # not empty
|
||||
return False # some permissions in b not in a
|
||||
return True
|
||||
|
||||
def _update(self, auth_id, description):
|
||||
ra = requests.patch(
|
||||
"{base}/api/v2/authorizations/{auth_id}".format(
|
||||
base=self.base, auth_id=auth_id),
|
||||
headers=self.h, json={"description": description})
|
||||
ra.raise_for_status()
|
||||
return ra
|
||||
|
||||
def _create(self, data):
|
||||
ra = requests.post("{base}/api/v2/authorizations".format(
|
||||
base=self.base),
|
||||
headers=self.h, json=data)
|
||||
ra.raise_for_status()
|
||||
self.result_token = ra.json()
|
||||
return ra
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
result = dict(changed=False, message="")
|
||||
module = AnsibleModule(
|
||||
argument_spec=dict(
|
||||
base=dict(type="str", required=True),
|
||||
org=dict(type="str", required=True),
|
||||
auth_token=dict(type="str", required=True),
|
||||
key=dict(type="str", required=True),
|
||||
description=dict(type="str", default=""),
|
||||
permissions=dict(type="list", elements='dict', options=dict(
|
||||
action=dict(type='str', choices=['read', 'write'],
|
||||
required=True),
|
||||
resource=dict(type='dict', options=dict(
|
||||
id=dict(type='str'),
|
||||
name=dict(type='str'),
|
||||
org=dict(type='str'),
|
||||
orgID=dict(type='str'),
|
||||
type=dict(type='str', required=True, choices=[
|
||||
"authorizations", "buckets", "dashboards", "orgs",
|
||||
"sources", "tasks", "telegrafs", "users", "variables",
|
||||
"scrapers", "secrets", "labels", "views", "documents",
|
||||
"notificationRules", "notificationEndpoints", "checks",
|
||||
"dbrp", "flows", "annotations", "functions"]),
|
||||
), required=True),
|
||||
), required=True),
|
||||
force=dict(type="bool", default=False),
|
||||
),
|
||||
supports_check_mode=True
|
||||
)
|
||||
h = {"Authorization": "Token {token}".format(
|
||||
token=module.params["auth_token"])}
|
||||
t = Token(module.params["base"], h, {
|
||||
"org_id": get_org_id(module.params["base"], module.params["org"], h),
|
||||
"key": module.params["key"],
|
||||
"perms": module.params["permissions"],
|
||||
"description": module.params["description"]})
|
||||
|
||||
changed = t.check()
|
||||
if t.result_token:
|
||||
result['token'] = t.result_token["token"]
|
||||
result['changed'] = changed
|
||||
|
||||
if module.check_mode:
|
||||
module.exit_json(**result)
|
||||
|
||||
if changed or module.params["force"]:
|
||||
t.run()
|
||||
result['token'] = t.result_token["token"]
|
||||
module.exit_json(**result)
|
||||
@@ -1,34 +0,0 @@
|
||||
- file:
|
||||
path: "/container/docker-images/cobald.{{cobald_image_tag|default('latest')}}/"
|
||||
state: directory
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
|
||||
- template:
|
||||
src: cobald.Dockerfile
|
||||
dest: "/container/docker-images/cobald.{{cobald_image_tag|default('latest')}}/Dockerfile"
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
register: cobald_cp_dockerfile
|
||||
|
||||
- copy:
|
||||
src: "{{item}}"
|
||||
dest: "/container/docker-images/cobald.{{cobald_image_tag|default('latest')}}/{{item}}"
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
mode: 0755
|
||||
with_items:
|
||||
- cobald-entrypoint.sh
|
||||
- init-cobaldmodules.sh
|
||||
register: cobald_cp_files
|
||||
|
||||
- docker_image:
|
||||
name: "cobald"
|
||||
tag: "{{cobald_image_tag|default('latest')}}"
|
||||
# pull: False
|
||||
build:
|
||||
pull: False
|
||||
path: "/container/docker-images/cobald.{{cobald_image_tag|default('latest')}}/"
|
||||
source: build
|
||||
force_source: "{{cobald_cp_dockerfile.changed or cobald_cp_files.changed}}"
|
||||
|
||||
@@ -1,11 +0,0 @@
|
||||
- block:
|
||||
- docker_container_info:
|
||||
name: "{{ container_name | mandatory }}"
|
||||
register: cobald_container_info
|
||||
|
||||
- set_fact:
|
||||
cobald_container_hostname: |-
|
||||
{{cobald_container_info.container.Config.Hostname | default('cobald-'+
|
||||
lookup('password', '/dev/null chars=ascii_lowercase length=6')) }}
|
||||
when: cobald_container_hostname is not defined
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
- name: create influx token for grafana
|
||||
influx_token:
|
||||
base: "http://localhost:{{influx_pubport}}"
|
||||
org: "my-org"
|
||||
auth_token: "{{influx_admin_token}}"
|
||||
description: grafana read access
|
||||
key: grafana
|
||||
permissions:
|
||||
- action: read
|
||||
resource:
|
||||
type: buckets
|
||||
register: influx_grafana_token
|
||||
|
||||
- name: run grafana
|
||||
docker_container:
|
||||
name: ed-grafana
|
||||
image: docker.io/grafana/grafana:7.5.7
|
||||
hostname: ed-grafana
|
||||
domainname: cobald.local
|
||||
networks:
|
||||
- name: "{{cobald_docker_network}}"
|
||||
networks_cli_compatible: True
|
||||
published_ports:
|
||||
- "3000:3000"
|
||||
state: started
|
||||
detach: True
|
||||
cleanup: True
|
||||
|
||||
- wait_for:
|
||||
host: localhost
|
||||
port: 3000
|
||||
|
||||
- community.grafana.grafana_datasource:
|
||||
grafana_url: http://localhost:3000
|
||||
grafana_user: admin
|
||||
grafana_password: admin
|
||||
name: InfluxDB
|
||||
ds_type: influxdb
|
||||
ds_url: "{{influx_url}}"
|
||||
additional_json_data:
|
||||
defaultBucket: "{{influx_bucket}}"
|
||||
organization: "{{influx_org}}"
|
||||
version: Flux
|
||||
additional_secure_json_data:
|
||||
token: "{{influx_grafana_token.token}}"
|
||||
|
||||
- community.grafana.grafana_dashboard:
|
||||
grafana_url: http://localhost:3000
|
||||
grafana_user: admin
|
||||
grafana_password: admin
|
||||
state: present
|
||||
commit_message: updated by ansible
|
||||
overwrite: yes
|
||||
json_data: "{{lookup('file', 'grafana-dashboard.json')|from_json}}"
|
||||
@@ -1,76 +0,0 @@
|
||||
- name: run influxdb in docker container
|
||||
docker_container:
|
||||
name: ed-influxdb
|
||||
image: docker.io/library/influxdb:2.0
|
||||
hostname: "{{cobald_influx_hostname}}"
|
||||
domainname: "{{cobald_domainname}}"
|
||||
networks:
|
||||
- name: "{{ cobald_docker_network }}"
|
||||
networks_cli_compatible: True
|
||||
published_ports:
|
||||
- "{{influx_pubport}}:8086"
|
||||
volumes:
|
||||
- "ed-influxdb-data:/var/lib/influxdb2"
|
||||
- "ed-influxdb-config:/etc/influxdb2"
|
||||
- "/container/volumes/influxdb-backup/:/backup"
|
||||
env:
|
||||
DOCKER_INFLUXDB_INIT_MODE: setup
|
||||
DOCKER_INFLUXDB_INIT_USERNAME: "{{influx_admin_user}}"
|
||||
DOCKER_INFLUXDB_INIT_PASSWORD: "{{influx_admin_pw}}"
|
||||
DOCKER_INFLUXDB_INIT_ORG: "{{influx_org}}"
|
||||
DOCKER_INFLUXDB_INIT_BUCKET: my-bucket
|
||||
state: started
|
||||
detach: True
|
||||
cleanup: True
|
||||
|
||||
- name: add ansible connection to influxdb container
|
||||
add_host:
|
||||
name: ed-influxdb
|
||||
ansible_connection: docker
|
||||
ansible_docker_extra_args: "-H=ssh://{{ansible_host}}"
|
||||
changed_when: False
|
||||
|
||||
- name: wait for influx to run
|
||||
raw: until curl http://localhost:8086 ; do sleep 1 ; done
|
||||
changed_when: False
|
||||
delegate_to: ed-influxdb
|
||||
|
||||
- name: fetch influxdb auth token
|
||||
raw: influx auth list --user my-user --hide-headers --json
|
||||
register: influx_token_fetch
|
||||
changed_when: False
|
||||
delegate_to: ed-influxdb
|
||||
|
||||
- name: set influxdb admin token
|
||||
set_fact:
|
||||
influx_admin_token:
|
||||
"{{(influx_token_fetch.stdout | from_json | first).token}}"
|
||||
|
||||
- name: create influxdb bucket for cobald
|
||||
influx_bucket:
|
||||
base: "http://localhost:{{influx_pubport}}"
|
||||
org: "my-org"
|
||||
auth_token: "{{influx_admin_token}}"
|
||||
name: "{{influx_bucket}}"
|
||||
|
||||
- name: create influxdb dashboard
|
||||
influx_dashboard:
|
||||
base: "http://localhost:{{influx_pubport}}"
|
||||
org: "my-org"
|
||||
auth_token: "{{influx_admin_token}}"
|
||||
data: "{{lookup('file', 'influxdb-dashboard-cobald.json')}}"
|
||||
when: influxdb_dashboard | default(True)
|
||||
|
||||
- name: create influxdb write access token for telegraf
|
||||
influx_token:
|
||||
base: "http://localhost:{{influx_pubport}}"
|
||||
org: "my-org"
|
||||
auth_token: "{{influx_admin_token}}"
|
||||
description: cobald tardis telegraf monitoring plugin
|
||||
key: telegraf_cobaldtardis
|
||||
permissions:
|
||||
- action: write
|
||||
resource:
|
||||
type: buckets
|
||||
name: "{{influx_bucket}}"
|
||||
register: influx_telegraf_token
|
||||
@@ -1,149 +0,0 @@
|
||||
- include_vars: cobald-slurm.yml
|
||||
when: cobald_slurm | default(False)
|
||||
tags: always
|
||||
|
||||
- name: build cobald:slurm docker image
|
||||
include_role:
|
||||
name: slurm
|
||||
tasks_from: dockerimage
|
||||
vars:
|
||||
slurm_image_prefix: cobald
|
||||
image_name: "{{cobald_image_tag}}"
|
||||
dockerfile: "{{ lookup('template', 'cobald.Dockerfile') }}"
|
||||
files_list:
|
||||
- cobald-entrypoint.sh
|
||||
- init-cobaldmodules.sh
|
||||
- start-drone
|
||||
- 28-sync-container-slurmd
|
||||
files: "
|
||||
{%- set files = [] -%} {%- for i in files_list -%}
|
||||
{%- set files = files.append(
|
||||
{ 'dest': i, 'content': lookup('file', i) }) -%}
|
||||
{%- endfor %}{{ files }}"
|
||||
when: cobald_slurm | default(False)
|
||||
|
||||
- name: build generic cobald docker image
|
||||
include_tasks: dockerimage-generic.yml
|
||||
when: not (cobald_slurm | default(False))
|
||||
|
||||
- name: make cobald data volume
|
||||
file:
|
||||
path: "/container/volumes/cobald/"
|
||||
state: directory
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
|
||||
- name: copy cobald config
|
||||
copy:
|
||||
src: cobald-config/
|
||||
dest: "~{{unpriv_user}}/cobald/"
|
||||
force: False
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
mode: "0644"
|
||||
|
||||
- name: ensure network for cobald container exists
|
||||
docker_network:
|
||||
name: "{{cobald_docker_network}}"
|
||||
state: present
|
||||
|
||||
# docker run -v $(pwd)/cobald-config-host:/etc/cobald -v $(pwd)/cobald:/cobald --rm -it cobald bash
|
||||
|
||||
- name: install git
|
||||
yum:
|
||||
name: git
|
||||
state: present
|
||||
|
||||
- name: make directories for cobald configuration and modules
|
||||
file:
|
||||
path: "{{item}}"
|
||||
owner: "{{unpriv_user}}"
|
||||
group: "{{unpriv_user}}"
|
||||
mode: "a=rx,u=rwx"
|
||||
state: directory
|
||||
with_items:
|
||||
- "~{{unpriv_user}}/cobald/modules"
|
||||
- "~{{unpriv_user}}/cobald"
|
||||
|
||||
- name: clone cobald code from git
|
||||
git:
|
||||
repo: https://github.com/thoto/cobald
|
||||
dest: "~{{unpriv_user}}/cobald-src"
|
||||
version: bugfix/mixed_construction_methods
|
||||
update: no # FIXME
|
||||
become: yes
|
||||
become_user: "{{unpriv_user}}"
|
||||
register: cobald_git_pull
|
||||
|
||||
- name: clone tardis code from git
|
||||
git:
|
||||
repo: https://github.com/MatterMiners/tardis
|
||||
dest: "~{{unpriv_user}}/tardis-src"
|
||||
version: master
|
||||
update: no # FIXME
|
||||
become: yes
|
||||
become_user: "{{unpriv_user}}"
|
||||
register: tardis_git_pull
|
||||
|
||||
- name: "get unpriv_user {{unpriv_user}} uid and gid"
|
||||
getent:
|
||||
database: passwd
|
||||
key: "{{unpriv_user}}"
|
||||
|
||||
- name: run pip install on cobald and tardis
|
||||
docker_container:
|
||||
image: "cobald:{{cobald_image_tag|default('latest')}}"
|
||||
name: "cobald-src-{{item.name}}-install"
|
||||
volumes:
|
||||
- "~{{unpriv_user}}/{{item.name}}-src:/usr/local/src/{{item.name}}:rw"
|
||||
state: started
|
||||
detach: False
|
||||
cleanup: True
|
||||
user: "{{getent_passwd[unpriv_user][1]}}:{{getent_passwd[unpriv_user][2]}}"
|
||||
entrypoint: ""
|
||||
command: |
|
||||
bash -c 'HOME=/tmp pip3 install --editable /usr/local/src/{{item.name}}'
|
||||
with_items:
|
||||
- name: cobald
|
||||
run: "{{cobald_git_pull.changed}}"
|
||||
- name: tardis
|
||||
run: "{{tardis_git_pull.changed}}"
|
||||
when: item.run
|
||||
|
||||
- import_tasks: telegraf.yml
|
||||
|
||||
- name: get cobald hostname
|
||||
include_tasks: facts.yml
|
||||
when: cobald_container_hostname is not defined
|
||||
|
||||
- name: build singularity container
|
||||
include_tasks:
|
||||
file: singularity.yml
|
||||
apply:
|
||||
tags: singularity
|
||||
tags: singularity
|
||||
|
||||
- name: run cobald container
|
||||
docker_container:
|
||||
name: "{{ container_name | default('cobald') }}"
|
||||
image: "cobald:{{cobald_image_tag|default('latest')}}"
|
||||
hostname: "{{cobald_container_hostname}}"
|
||||
domainname: "{{ cobald_domainname | default('cobald.local')}}"
|
||||
volumes: "{{default_mounts + cobald_mounts }}"
|
||||
networks:
|
||||
- name: "{{cobald_docker_network}}"
|
||||
networks_cli_compatible: True
|
||||
state: started
|
||||
detach: True
|
||||
cleanup: True
|
||||
interactive: True
|
||||
# command: python3 -m cobald.daemon /etc/cobald/config.yaml
|
||||
vars:
|
||||
default_mounts: "{{cobald_slurm_mounts | default([])}}"
|
||||
cobald_mounts:
|
||||
- "~{{unpriv_user}}/cobald:/etc/cobald"
|
||||
# - /container/volumes/cobald:/etc/cobald:ro
|
||||
- "/container/docker-images/sing-slurmd/build/:/slurm-singimage/:ro"
|
||||
- "~{{unpriv_user}}/cobald/modules:/usr/local/src/cobaldmodules"
|
||||
- "~{{unpriv_user}}/cobald-src:/usr/local/src/cobald:ro"
|
||||
- "~{{unpriv_user}}/tardis-src:/usr/local/src/tardis:ro"
|
||||
@@ -1,48 +0,0 @@
|
||||
- name: setup singularity
|
||||
import_role: name="singularity"
|
||||
tags: singularity
|
||||
|
||||
- name: make singularity image build directory
|
||||
file:
|
||||
state: directory
|
||||
path: "{{item}}"
|
||||
owner: "{{unpriv_user}}"
|
||||
group: "docker"
|
||||
mode: "0755"
|
||||
loop:
|
||||
- /container/docker-images/sing-slurmd
|
||||
- /container/docker-images/sing-slurmd/cache
|
||||
- /container/docker-images/sing-slurmd/build
|
||||
|
||||
- name: copy slurm singularity container files
|
||||
copy:
|
||||
src: "{{item}}"
|
||||
dest: "/container/docker-images/sing-slurmd/{{item}}"
|
||||
owner: "{{unpriv_user}}"
|
||||
group: "docker"
|
||||
loop:
|
||||
- slurm-slurmd.def
|
||||
- 31-slurmd-configless
|
||||
- cgroup.conf.noautomount
|
||||
register: cobald_copy_sing_files
|
||||
|
||||
- name: remove old container
|
||||
file:
|
||||
path: /container/docker-images/sing-slurmd/build/slurmd.sif
|
||||
state: absent
|
||||
when: cobald_copy_sing_files.changed
|
||||
|
||||
- name: build container
|
||||
shell:
|
||||
chdir: /container/docker-images/sing-slurmd/
|
||||
cmd: SINGULARITY_TMPDIR=/container/docker-images/sing-slurmd/cache
|
||||
singularity build --disable-cache
|
||||
/container/docker-images/sing-slurmd/build/slurmd.sif
|
||||
/container/docker-images/sing-slurmd/slurm-slurmd.def
|
||||
creates: /container/docker-images/sing-slurmd/build/slurmd.sif
|
||||
register: cobald_sing_build
|
||||
|
||||
- debug: msg="{{[cobald_sing_build.stdout, cobald_sing_build.stderr]}}"
|
||||
tags: [ never, debug ]
|
||||
|
||||
# TODO: trigger copy in cobald container when slurmd.sif rebuilt
|
||||
@@ -1,69 +0,0 @@
|
||||
- name: setup directories for telegraf
|
||||
file:
|
||||
path: "/container/{{item}}/telegraf/"
|
||||
state: directory
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
loop:
|
||||
- docker-images
|
||||
- volumes
|
||||
|
||||
- name: copy telegraf Dockerfile
|
||||
copy:
|
||||
src: telegraf.Dockerfile
|
||||
dest: /container/docker-images/telegraf/Dockerfile
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
register: cobald_cp_telegraf_dockerfile
|
||||
|
||||
- name: copy telegraf repo file
|
||||
copy: # telegraf is found in influxdb repo
|
||||
src: influxdb.repo
|
||||
dest: /container/docker-images/telegraf/influxdb.repo
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
|
||||
- name: docker image for telegraf
|
||||
docker_image:
|
||||
name: "ed-telegraf"
|
||||
build:
|
||||
pull: False
|
||||
path: "/container/docker-images/telegraf/"
|
||||
source: build
|
||||
force_source: "{{cobald_cp_telegraf_dockerfile.changed}}"
|
||||
|
||||
- import_tasks: influxdb.yml
|
||||
tags: influxdb
|
||||
|
||||
- name: generate telegraf config
|
||||
template:
|
||||
src: telegraf.conf.j2
|
||||
dest: /container/volumes/telegraf/telegraf.conf
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
vars:
|
||||
influx_token: "{{influx_telegraf_token.token}}"
|
||||
influx_url: "http://{{cobald_influx_hostname}}:8086"
|
||||
register: telegraf_config_gen
|
||||
|
||||
- name: run telegraf container
|
||||
docker_container:
|
||||
name: ed-telegraf
|
||||
image: ed-telegraf
|
||||
hostname: telegraf
|
||||
domainname: "{{ cobald_domainname }}"
|
||||
networks:
|
||||
- name: "{{ cobald_docker_network }}"
|
||||
aliases: ["ed-telegraf"]
|
||||
volumes:
|
||||
- "/container/volumes/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro"
|
||||
state: started
|
||||
recreate: "{{ telegraf_config_gen.changed | default(False) | bool }}"
|
||||
detach: True
|
||||
# cleanup: True
|
||||
networks_cli_compatible: True
|
||||
|
||||
- import_tasks: grafana.yml
|
||||
vars:
|
||||
influx_url: "http://{{cobald_influx_hostname}}:8086"
|
||||
tags: influxdb
|
||||
@@ -1,73 +0,0 @@
|
||||
FROM {{ cobald_docker_base_image | default("docker.io/library/centos:7") }}
|
||||
|
||||
RUN yum update -y && \
|
||||
yum install -y python3 git && pip3 install --upgrade pip && \
|
||||
yum clean all && rm -rf /var/cache/yum
|
||||
|
||||
ARG REPOCOBALD=https://github.com/MatterMiners/cobald
|
||||
ARG REPOTARDIS=https://github.com/MatterMiners/tardis
|
||||
|
||||
RUN git clone $REPOCOBALD /usr/local/src/cobald && \
|
||||
git clone $REPOTARDIS /usr/local/src/tardis
|
||||
|
||||
RUN mkdir /etc/cobald /var/log/cobald && \
|
||||
( getent passwd cobald > /dev/null || \
|
||||
useradd -m -d /var/lib/cobald --no-log-init --system cobald ) && \
|
||||
chown cobald:cobald /var/log/cobald
|
||||
|
||||
#RUN mkdir /cobald && python3 -m venv /cobald && source /cobald/bin/activate &&\
|
||||
# pip3 install --upgrade pip && pip3 install cobald
|
||||
|
||||
RUN mkdir /usr/local/src/cobaldmodules /usr/local/lib/cobaldmodules && \
|
||||
ln -s /usr/local/src/cobaldmodules/setup.py \
|
||||
/usr/local/lib/cobaldmodules/setup.py && \
|
||||
ln -s /usr/local/src/cobaldmodules/cobaldmodules \
|
||||
/usr/local/lib/cobaldmodules/cobaldmodules && \
|
||||
chown cobald:cobald /usr/local/lib/cobaldmodules
|
||||
|
||||
RUN pip3 install --editable /usr/local/src/cobald && \
|
||||
pip3 install --editable /usr/local/src/cobald[contrib]
|
||||
|
||||
RUN pip3 install --editable /usr/local/src/tardis&& \
|
||||
pip3 install --editable /usr/local/src/tardis[contrib]
|
||||
|
||||
ENV PYTHONPATH=/usr/local/src/cobaldmodules
|
||||
|
||||
# pip3 install --editable .
|
||||
# pip3 install --editable .[contrib]
|
||||
# pip3 install --upgrade --editable /etc/cobald/modules/
|
||||
# su cobald -c "python3 -m cobald.daemon /etc/cobald/config.yaml"
|
||||
|
||||
VOLUME /usr/local/src/cobaldmodules
|
||||
|
||||
VOLUME /etc/cobald
|
||||
|
||||
RUN mkdir -p /usr/local/lib/entrypoints.d/
|
||||
|
||||
COPY init-cobaldmodules.sh /usr/local/lib/entrypoints.d/50-init-cobaldmodules.sh
|
||||
|
||||
RUN chmod 755 /usr/local/lib/entrypoints.d/50-init-cobaldmodules.sh
|
||||
|
||||
COPY start-drone /usr/local/bin/start-drone
|
||||
COPY 28-sync-container-slurmd /etc/docker-init.d/28-sync-container-slurmd
|
||||
RUN chmod 755 /usr/local/bin/start-drone /etc/docker-init.d/28-sync-container-slurmd
|
||||
|
||||
RUN echo -e "#!/bin/sh\npython3 -m cobald.daemon /etc/cobald/config.yaml" >> /etc/docker-init.d/70-cobald && chmod 755 /etc/docker-init.d/70-cobald
|
||||
|
||||
{% if cobald_docker_default_command | default(True) -%}
|
||||
COPY cobald-entrypoint.sh /usr/local/sbin/cobald-entrypoint.sh
|
||||
|
||||
RUN chmod 755 /usr/local/sbin/cobald-entrypoint.sh
|
||||
|
||||
ENTRYPOINT [ "/usr/local/sbin/cobald-entrypoint.sh" ]
|
||||
|
||||
RUN yum -y install iproute &&\
|
||||
yum clean all && rm -rf /var/cache/yum
|
||||
|
||||
USER cobald
|
||||
|
||||
STOPSIGNAL SIGINT
|
||||
|
||||
# CMD "python3 -m cobald.daemon /etc/cobald/config.yaml"
|
||||
CMD /etc/docker-init.d/60-cobald
|
||||
{%- endif %}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,12 +0,0 @@
|
||||
cobald_image_tag: slurm
|
||||
cobald_docker_base_image: "{{slurm.base_image}}"
|
||||
cobald_docker_default_command: False
|
||||
cobald_docker_network: "{{slurm.network}}"
|
||||
cobald_domainname: "{{slurm.domain}}"
|
||||
cobald_slurm_mounts: "{{slurm.mounts}}"
|
||||
#- /container/volumes/slurm/:/etc/slurm/:rw
|
||||
##- "{{slurm_cfg_path | mandatory}}:/etc/slurm/:rw"
|
||||
#- /container/volumes/munge/munge.key:/etc/munge/munge.key:rw
|
||||
## - "{{slurm_munge_path | mandatory}}:/etc/munge/munge.key:rw"
|
||||
#- slurm-shared:/shared/:rw
|
||||
## - "{{slurm_shared_path | mandatory}}:{{slurm_shared_target | default('/shared')}}:rw"
|
||||
@@ -1,2 +0,0 @@
|
||||
cobald_docker_network: "{{docker_network}}"
|
||||
cobald_influx_hostname: "ed-influxdb"
|
||||
@@ -1,3 +0,0 @@
|
||||
use ROLE:centralmanager
|
||||
# ALLOW_WRITE per default on * -> following has no effect yet
|
||||
ALLOW_WRITE_COLLECTOR=$(ALLOW_WRITE) condor-exec.htc.local condor-sub.htc.local
|
||||
@@ -1,92 +0,0 @@
|
||||
######################################################################
|
||||
##
|
||||
## condor_config
|
||||
##
|
||||
## This is the global configuration file for condor. This is where
|
||||
## you define where the local config file is. Any settings
|
||||
## made here may potentially be overridden in the local configuration
|
||||
## file. KEEP THAT IN MIND! To double-check that a variable is
|
||||
## getting set from the configuration file that you expect, use
|
||||
## condor_config_val -v <variable name>
|
||||
##
|
||||
## condor_config.annotated is a more detailed sample config file
|
||||
##
|
||||
## Unless otherwise specified, settings that are commented out show
|
||||
## the defaults that are used if you don't define a value. Settings
|
||||
## that are defined here MUST BE DEFINED since they have no default
|
||||
## value.
|
||||
##
|
||||
######################################################################
|
||||
|
||||
## Where have you installed the bin, sbin and lib condor directories?
|
||||
RELEASE_DIR = /usr
|
||||
|
||||
## Where is the local condor directory for each host? This is where the local config file(s), logs and
|
||||
## spool/execute directories are located. this is the default for Linux and Unix systems.
|
||||
LOCAL_DIR = /var
|
||||
|
||||
## Where is the machine-specific local config file for each host?
|
||||
# LOCAL_CONFIG_FILE = /etc/condor/condor_config.local
|
||||
LOCAL_CONFIG_FILE = /etc/condor/condor_config_$(HOSTNAME).local
|
||||
## If your configuration is on a shared file system, then this might be a better default
|
||||
#LOCAL_CONFIG_FILE = $(RELEASE_DIR)/etc/$(HOSTNAME).local
|
||||
## If the local config file is not present, is it an error? (WARNING: This is a potential security issue.)
|
||||
REQUIRE_LOCAL_CONFIG_FILE = false
|
||||
|
||||
## The normal way to do configuration with RPMs is to read all of the
|
||||
## files in a given directory that don't match a regex as configuration files.
|
||||
## Config files are read in lexicographic order.
|
||||
LOCAL_CONFIG_DIR = /etc/condor/config.d
|
||||
#LOCAL_CONFIG_DIR_EXCLUDE_REGEXP = ^((\..*)|(.*~)|(#.*)|(.*\.rpmsave)|(.*\.rpmnew))$
|
||||
|
||||
##
|
||||
## Do NOT use host-based security by default.
|
||||
##
|
||||
## This was the default for the 8.8 series (and earlier), but it is
|
||||
## intrinsically insecure. To make the 9.0 series secure by default, we
|
||||
## commented it out.
|
||||
##
|
||||
## You should seriously consider improving your security configuration.
|
||||
##
|
||||
## To continue to use your old security configuration, knowing that it is
|
||||
## insecure, add the line 'use SECURITY : HOST_BASED' to your local
|
||||
## configuration directory. Don't just uncomment the final line in this
|
||||
## comment block; changes in this file may be lost during your next upgrade.
|
||||
## The following shell command will make the change on most Linux systems.
|
||||
##
|
||||
## echo 'use SECURITY : HOST_BASED' >> $(condor_config_val LOCAL_CONFIG_DIR)/00-insecure.config
|
||||
##
|
||||
|
||||
## To expand your condor pool beyond a single host, set ALLOW_WRITE to match all of the hosts
|
||||
#ALLOW_WRITE = *.cs.wisc.edu
|
||||
## FLOCK_FROM defines the machines that grant access to your pool via flocking. (i.e. these machines can join your pool).
|
||||
#FLOCK_FROM =
|
||||
## FLOCK_TO defines the central managers that your schedd will advertise itself to (i.e. these pools will give matches to your schedd).
|
||||
#FLOCK_TO = condor.cs.wisc.edu, cm.example.edu
|
||||
|
||||
##--------------------------------------------------------------------
|
||||
## Values set by the rpm patch script:
|
||||
##--------------------------------------------------------------------
|
||||
|
||||
## For Unix machines, the path and file name of the file containing
|
||||
## the pool password for password authentication.
|
||||
#SEC_PASSWORD_FILE = $(LOCAL_DIR)/lib/condor/pool_password
|
||||
|
||||
## Pathnames
|
||||
RUN = $(LOCAL_DIR)/run/condor
|
||||
LOG = $(LOCAL_DIR)/log/condor
|
||||
LOCK = $(LOCAL_DIR)/lock/condor
|
||||
SPOOL = $(LOCAL_DIR)/lib/condor/spool
|
||||
EXECUTE = $(LOCAL_DIR)/lib/condor/execute
|
||||
BIN = $(RELEASE_DIR)/bin
|
||||
LIB = $(RELEASE_DIR)/lib64/condor
|
||||
INCLUDE = $(RELEASE_DIR)/include/condor
|
||||
SBIN = $(RELEASE_DIR)/sbin
|
||||
LIBEXEC = $(RELEASE_DIR)/libexec/condor
|
||||
SHARE = $(RELEASE_DIR)/share/condor
|
||||
|
||||
PROCD_ADDRESS = $(RUN)/procd_pipe
|
||||
|
||||
JAVA_CLASSPATH_DEFAULT = $(SHARE) .
|
||||
|
||||
## Install the minicondor package to run HTCondor on a single node
|
||||
@@ -1,64 +0,0 @@
|
||||
|
||||
HostKey _INSERT_HOST_KEY_
|
||||
AuthorizedKeysFile _INSERT_AUTHORIZED_KEYS_FILE_
|
||||
|
||||
# The following option is not supported by all recent versions of OpenSSH,
|
||||
# so instead we rely on injection of the shell setup command in the authorized
|
||||
# keys file.
|
||||
#ForceCommand _INSERT_FORCE_COMMAND_
|
||||
|
||||
# as a convenience to users, allow remote setting of environment
|
||||
# since sshd is running as the job uid, there isn't really a security concern
|
||||
AcceptEnv *
|
||||
|
||||
Subsystem sftp /usr/libexec/openssh/sftp-server
|
||||
|
||||
# prevent sshd from freaking out about reading files from inside
|
||||
# a tmp-like directory
|
||||
StrictModes no
|
||||
|
||||
# Rejection by tcp wrappers is not logged at INFO or VERBOSE log levels,
|
||||
# so to make diagnosis of problems easier, we use DEBUG.
|
||||
LogLevel DEBUG
|
||||
|
||||
X11Forwarding yes
|
||||
|
||||
# By default, OpenSSH uses the ipv6 loopback even if ipv6 is disabled in the
|
||||
# kernel. This forces OpenSSH to use the "local" network instead
|
||||
X11UseLocalhost no
|
||||
|
||||
# The following settings are recommended for good security.
|
||||
# In particular, only FIPS 140-2 algorithms are used.
|
||||
# URLs for extra information re FIPS security compliance:
|
||||
# https://rhel7stig.readthedocs.io/en/latest/
|
||||
# https://www.stigviewer.com/stig/red_hat_enterprise_linux_7/
|
||||
# https://people.redhat.com/swells/scap-security-guide/tables/table-rhel7-stig-manual.html
|
||||
|
||||
#-FIPS High Severity
|
||||
Protocol 2
|
||||
PermitEmptyPasswords no
|
||||
|
||||
#-FIPS Medium Severity
|
||||
# Note: Ciphers and MACs below will be incompatible with RHEL5 or earlier.
|
||||
Ciphers aes128-ctr,aes192-ctr,aes256-ctr
|
||||
MACs hmac-sha2-256,hmac-sha2-512
|
||||
PermitRootLogin no
|
||||
PermitUserEnvironment no
|
||||
GSSAPIAuthentication no
|
||||
KerberosAuthentication no
|
||||
HostbasedAuthentication no
|
||||
IgnoreRhosts yes
|
||||
IgnoreUserKnownHosts yes
|
||||
PrintLastLog yes
|
||||
UsePrivilegeSeparation sandbox
|
||||
Compression delayed
|
||||
|
||||
#-Recommended for security, but left out ssh_to_job config
|
||||
# because they provide minimal value and are likely to annoy
|
||||
# users or generate needless warnings in the ssh_to_job setting.
|
||||
#
|
||||
# ClientAliveInterval 600 # Note: condor_submit -i sets TMOUT
|
||||
# ClientAliveCountMax 0
|
||||
# banner=/etc/issue # Set to your warning banner
|
||||
# StrictModes yes # Can't set due to tmp-like directory
|
||||
# RhostsRSAAuthentication no # Obsolete Protocol version 1 option
|
||||
@@ -1,26 +0,0 @@
|
||||
##
|
||||
## Default security settings
|
||||
##
|
||||
## Host-based security was the default for the 8.8 series (and earlier).
|
||||
##
|
||||
## Host-based security assumes that all users on a machine are trusted.
|
||||
## For example, if host-based security trusts that a given machine can
|
||||
## run jobs, then any user who can start a process on that machine can
|
||||
## start a startd that can "steal" jobs from the system.
|
||||
#
|
||||
## To help make HTCondor secure by default, we removed host-based security
|
||||
## from the default configuration file
|
||||
## (/etc/condor/condor_config).
|
||||
##
|
||||
## New installations of HTCondor should be made using the get_htcondor tool,
|
||||
## which can automatically establish IDTOKENS-based security across a multi-
|
||||
## node pool. For existing installations, we recommend you
|
||||
## consider improving your security configuration.
|
||||
##
|
||||
## To continue to use your old security configuration,
|
||||
## comment out the 'recommended' line below, and uncomment the
|
||||
## 'host_based' line.
|
||||
##
|
||||
|
||||
# use security : host_based
|
||||
use security : recommended_v9_0
|
||||
@@ -1 +0,0 @@
|
||||
CONDOR_HOST = condor-cm.htc.local
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1 +0,0 @@
|
||||
use ROLE:execute
|
||||
@@ -1 +0,0 @@
|
||||
use ROLE:submit
|
||||
@@ -1,11 +0,0 @@
|
||||
FROM docker.io/library/centos:7
|
||||
|
||||
RUN yum install -y https://research.cs.wisc.edu/htcondor/repo/8.9/htcondor-release-current.el7.noarch.rpm && \
|
||||
yum install --nogpgcheck -y condor && \
|
||||
yum install -y less && \
|
||||
yum clean all
|
||||
|
||||
RUN yum install -y iproute bind-utils nmap-ncat net-tools && \
|
||||
yum clean all
|
||||
|
||||
CMD bash -c 'cat <({ condor_master -f & tail --retry --pid $! -f /var/log/condor/MasterLog & })'
|
||||
@@ -1,793 +0,0 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright: (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
|
||||
# Copyright: (c) 2017, Ansible Project
|
||||
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
|
||||
|
||||
from __future__ import absolute_import, division, print_function
|
||||
__metaclass__ = type
|
||||
|
||||
ANSIBLE_METADATA = {'metadata_version': '1.1',
|
||||
'status': ['stableinterface'],
|
||||
'supported_by': 'core'}
|
||||
|
||||
DOCUMENTATION = r'''
|
||||
---
|
||||
module: copy
|
||||
version_added: historical
|
||||
short_description: Copy files to remote locations
|
||||
description:
|
||||
- The C(copy) module copies a file from the local or remote machine to a location on the remote machine.
|
||||
- Use the M(fetch) module to copy files from remote locations to the local box.
|
||||
- If you need variable interpolation in copied files, use the M(template) module. Using a variable in the C(content)
|
||||
field will result in unpredictable output.
|
||||
- For Windows targets, use the M(win_copy) module instead.
|
||||
options:
|
||||
src:
|
||||
description:
|
||||
- Local path to a file to copy to the remote server.
|
||||
- This can be absolute or relative.
|
||||
- If path is a directory, it is copied recursively. In this case, if path ends
|
||||
with "/", only inside contents of that directory are copied to destination.
|
||||
Otherwise, if it does not end with "/", the directory itself with all contents
|
||||
is copied. This behavior is similar to the C(rsync) command line tool.
|
||||
type: path
|
||||
content:
|
||||
description:
|
||||
- When used instead of C(src), sets the contents of a file directly to the specified value.
|
||||
- Works only when C(dest) is a file. Creates the file if it does not exist.
|
||||
- For advanced formatting or if C(content) contains a variable, use the M(template) module.
|
||||
type: str
|
||||
version_added: '1.1'
|
||||
dest:
|
||||
description:
|
||||
- Remote absolute path where the file should be copied to.
|
||||
- If C(src) is a directory, this must be a directory too.
|
||||
- If C(dest) is a non-existent path and if either C(dest) ends with "/" or C(src) is a directory, C(dest) is created.
|
||||
- If I(dest) is a relative path, the starting directory is determined by the remote host.
|
||||
- If C(src) and C(dest) are files, the parent directory of C(dest) is not created and the task fails if it does not already exist.
|
||||
type: path
|
||||
required: yes
|
||||
backup:
|
||||
description:
|
||||
- Create a backup file including the timestamp information so you can get the original file back if you somehow clobbered it incorrectly.
|
||||
type: bool
|
||||
default: no
|
||||
version_added: '0.7'
|
||||
force:
|
||||
description:
|
||||
- Influence whether the remote file must always be replaced.
|
||||
- If C(yes), the remote file will be replaced when contents are different than the source.
|
||||
- If C(no), the file will only be transferred if the destination does not exist.
|
||||
- Alias C(thirsty) has been deprecated and will be removed in 2.13.
|
||||
type: bool
|
||||
default: yes
|
||||
aliases: [ thirsty ]
|
||||
version_added: '1.1'
|
||||
mode:
|
||||
description:
|
||||
- The permissions of the destination file or directory.
|
||||
- For those used to C(/usr/bin/chmod) remember that modes are actually octal numbers.
|
||||
You must either add a leading zero so that Ansible's YAML parser knows it is an octal number
|
||||
(like C(0644) or C(01777))or quote it (like C('644') or C('1777')) so Ansible receives a string
|
||||
and can do its own conversion from string into number. Giving Ansible a number without following
|
||||
one of these rules will end up with a decimal number which will have unexpected results.
|
||||
- As of Ansible 1.8, the mode may be specified as a symbolic mode (for example, C(u+rwx) or C(u=rw,g=r,o=r)).
|
||||
- As of Ansible 2.3, the mode may also be the special string C(preserve).
|
||||
- C(preserve) means that the file will be given the same permissions as the source file.
|
||||
type: path
|
||||
directory_mode:
|
||||
description:
|
||||
- When doing a recursive copy set the mode for the directories.
|
||||
- If this is not set we will use the system defaults.
|
||||
- The mode is only set on directories which are newly created, and will not affect those that already existed.
|
||||
type: raw
|
||||
version_added: '1.5'
|
||||
remote_src:
|
||||
description:
|
||||
- Influence whether C(src) needs to be transferred or already is present remotely.
|
||||
- If C(no), it will search for C(src) at originating/master machine.
|
||||
- If C(yes) it will go to the remote/target machine for the C(src).
|
||||
- C(remote_src) supports recursive copying as of version 2.8.
|
||||
- C(remote_src) only works with C(mode=preserve) as of version 2.6.
|
||||
type: bool
|
||||
default: no
|
||||
version_added: '2.0'
|
||||
follow:
|
||||
description:
|
||||
- This flag indicates that filesystem links in the destination, if they exist, should be followed.
|
||||
type: bool
|
||||
default: no
|
||||
version_added: '1.8'
|
||||
local_follow:
|
||||
description:
|
||||
- This flag indicates that filesystem links in the source tree, if they exist, should be followed.
|
||||
type: bool
|
||||
default: yes
|
||||
version_added: '2.4'
|
||||
checksum:
|
||||
description:
|
||||
- SHA1 checksum of the file being transferred.
|
||||
- Used to validate that the copy of the file was successful.
|
||||
- If this is not provided, ansible will use the local calculated checksum of the src file.
|
||||
type: str
|
||||
version_added: '2.5'
|
||||
extends_documentation_fragment:
|
||||
- decrypt
|
||||
- files
|
||||
- validate
|
||||
notes:
|
||||
- The M(copy) module recursively copy facility does not scale to lots (>hundreds) of files.
|
||||
seealso:
|
||||
- module: assemble
|
||||
- module: fetch
|
||||
- module: file
|
||||
- module: synchronize
|
||||
- module: template
|
||||
- module: win_copy
|
||||
author:
|
||||
- Ansible Core Team
|
||||
- Michael DeHaan
|
||||
'''
|
||||
|
||||
EXAMPLES = r'''
|
||||
- name: Copy file with owner and permissions
|
||||
copy:
|
||||
src: /srv/myfiles/foo.conf
|
||||
dest: /etc/foo.conf
|
||||
owner: foo
|
||||
group: foo
|
||||
mode: '0644'
|
||||
|
||||
- name: Copy file with owner and permission, using symbolic representation
|
||||
copy:
|
||||
src: /srv/myfiles/foo.conf
|
||||
dest: /etc/foo.conf
|
||||
owner: foo
|
||||
group: foo
|
||||
mode: u=rw,g=r,o=r
|
||||
|
||||
- name: Another symbolic mode example, adding some permissions and removing others
|
||||
copy:
|
||||
src: /srv/myfiles/foo.conf
|
||||
dest: /etc/foo.conf
|
||||
owner: foo
|
||||
group: foo
|
||||
mode: u+rw,g-wx,o-rwx
|
||||
|
||||
- name: Copy a new "ntp.conf file into place, backing up the original if it differs from the copied version
|
||||
copy:
|
||||
src: /mine/ntp.conf
|
||||
dest: /etc/ntp.conf
|
||||
owner: root
|
||||
group: root
|
||||
mode: '0644'
|
||||
backup: yes
|
||||
|
||||
- name: Copy a new "sudoers" file into place, after passing validation with visudo
|
||||
copy:
|
||||
src: /mine/sudoers
|
||||
dest: /etc/sudoers
|
||||
validate: /usr/sbin/visudo -csf %s
|
||||
|
||||
- name: Copy a "sudoers" file on the remote machine for editing
|
||||
copy:
|
||||
src: /etc/sudoers
|
||||
dest: /etc/sudoers.edit
|
||||
remote_src: yes
|
||||
validate: /usr/sbin/visudo -csf %s
|
||||
|
||||
- name: Copy using inline content
|
||||
copy:
|
||||
content: '# This file was moved to /etc/other.conf'
|
||||
dest: /etc/mine.conf
|
||||
|
||||
- name: If follow=yes, /path/to/file will be overwritten by contents of foo.conf
|
||||
copy:
|
||||
src: /etc/foo.conf
|
||||
dest: /path/to/link # link to /path/to/file
|
||||
follow: yes
|
||||
|
||||
- name: If follow=no, /path/to/link will become a file and be overwritten by contents of foo.conf
|
||||
copy:
|
||||
src: /etc/foo.conf
|
||||
dest: /path/to/link # link to /path/to/file
|
||||
follow: no
|
||||
'''
|
||||
|
||||
RETURN = r'''
|
||||
dest:
|
||||
description: Destination file/path
|
||||
returned: success
|
||||
type: str
|
||||
sample: /path/to/file.txt
|
||||
src:
|
||||
description: Source file used for the copy on the target machine
|
||||
returned: changed
|
||||
type: str
|
||||
sample: /home/httpd/.ansible/tmp/ansible-tmp-1423796390.97-147729857856000/source
|
||||
md5sum:
|
||||
description: MD5 checksum of the file after running copy
|
||||
returned: when supported
|
||||
type: str
|
||||
sample: 2a5aeecc61dc98c4d780b14b330e3282
|
||||
checksum:
|
||||
description: SHA1 checksum of the file after running copy
|
||||
returned: success
|
||||
type: str
|
||||
sample: 6e642bb8dd5c2e027bf21dd923337cbb4214f827
|
||||
backup_file:
|
||||
description: Name of backup file created
|
||||
returned: changed and if backup=yes
|
||||
type: str
|
||||
sample: /path/to/file.txt.2015-02-12@22:09~
|
||||
gid:
|
||||
description: Group id of the file, after execution
|
||||
returned: success
|
||||
type: int
|
||||
sample: 100
|
||||
group:
|
||||
description: Group of the file, after execution
|
||||
returned: success
|
||||
type: str
|
||||
sample: httpd
|
||||
owner:
|
||||
description: Owner of the file, after execution
|
||||
returned: success
|
||||
type: str
|
||||
sample: httpd
|
||||
uid:
|
||||
description: Owner id of the file, after execution
|
||||
returned: success
|
||||
type: int
|
||||
sample: 100
|
||||
mode:
|
||||
description: Permissions of the target, after execution
|
||||
returned: success
|
||||
type: str
|
||||
sample: 0644
|
||||
size:
|
||||
description: Size of the target, after execution
|
||||
returned: success
|
||||
type: int
|
||||
sample: 1220
|
||||
state:
|
||||
description: State of the target, after execution
|
||||
returned: success
|
||||
type: str
|
||||
sample: file
|
||||
'''
|
||||
|
||||
import errno
|
||||
import filecmp
|
||||
import grp
|
||||
import os
|
||||
import os.path
|
||||
import platform
|
||||
import pwd
|
||||
import shutil
|
||||
import stat
|
||||
import tempfile
|
||||
import traceback
|
||||
|
||||
from ansible.module_utils.basic import AnsibleModule
|
||||
from ansible.module_utils.common.process import get_bin_path
|
||||
from ansible.module_utils._text import to_bytes, to_native
|
||||
from ansible.module_utils.six import PY3
|
||||
|
||||
|
||||
# The AnsibleModule object
|
||||
module = None
|
||||
|
||||
|
||||
class AnsibleModuleError(Exception):
|
||||
def __init__(self, results):
|
||||
self.results = results
|
||||
|
||||
|
||||
# Once we get run_command moved into common, we can move this into a common/files module. We can't
|
||||
# until then because of the module.run_command() method. We may need to move it into
|
||||
# basic::AnsibleModule() until then but if so, make it a private function so that we don't have to
|
||||
# keep it for backwards compatibility later.
|
||||
def clear_facls(path):
|
||||
setfacl = get_bin_path('setfacl', True)
|
||||
# FIXME "setfacl -b" is available on Linux and FreeBSD. There is "setfacl -D e" on z/OS. Others?
|
||||
acl_command = [setfacl, '-b', path]
|
||||
b_acl_command = [to_bytes(x) for x in acl_command]
|
||||
rc, out, err = module.run_command(b_acl_command, environ_update=dict(LANG='C', LC_ALL='C', LC_MESSAGES='C'))
|
||||
if rc != 0:
|
||||
raise RuntimeError('Error running "{0}": stdout: "{1}"; stderr: "{2}"'.format(' '.join(b_acl_command), out, err))
|
||||
|
||||
|
||||
def split_pre_existing_dir(dirname):
|
||||
'''
|
||||
Return the first pre-existing directory and a list of the new directories that will be created.
|
||||
'''
|
||||
head, tail = os.path.split(dirname)
|
||||
b_head = to_bytes(head, errors='surrogate_or_strict')
|
||||
if head == '':
|
||||
return ('.', [tail])
|
||||
if not os.path.exists(b_head):
|
||||
if head == '/':
|
||||
raise AnsibleModuleError(results={'msg': "The '/' directory doesn't exist on this machine."})
|
||||
(pre_existing_dir, new_directory_list) = split_pre_existing_dir(head)
|
||||
else:
|
||||
return (head, [tail])
|
||||
new_directory_list.append(tail)
|
||||
return (pre_existing_dir, new_directory_list)
|
||||
|
||||
|
||||
def adjust_recursive_directory_permissions(pre_existing_dir, new_directory_list, module, directory_args, changed):
|
||||
'''
|
||||
Walk the new directories list and make sure that permissions are as we would expect
|
||||
'''
|
||||
|
||||
if new_directory_list:
|
||||
working_dir = os.path.join(pre_existing_dir, new_directory_list.pop(0))
|
||||
directory_args['path'] = working_dir
|
||||
changed = module.set_fs_attributes_if_different(directory_args, changed)
|
||||
changed = adjust_recursive_directory_permissions(working_dir, new_directory_list, module, directory_args, changed)
|
||||
return changed
|
||||
|
||||
|
||||
def chown_recursive(path, module):
|
||||
changed = False
|
||||
owner = module.params['owner']
|
||||
group = module.params['group']
|
||||
|
||||
if owner is not None:
|
||||
if not module.check_mode:
|
||||
for dirpath, dirnames, filenames in os.walk(path):
|
||||
owner_changed = module.set_owner_if_different(dirpath, owner, False)
|
||||
if owner_changed is True:
|
||||
changed = owner_changed
|
||||
for dir in [os.path.join(dirpath, d) for d in dirnames]:
|
||||
owner_changed = module.set_owner_if_different(dir, owner, False)
|
||||
if owner_changed is True:
|
||||
changed = owner_changed
|
||||
for file in [os.path.join(dirpath, f) for f in filenames]:
|
||||
owner_changed = module.set_owner_if_different(file, owner, False)
|
||||
if owner_changed is True:
|
||||
changed = owner_changed
|
||||
else:
|
||||
uid = pwd.getpwnam(owner).pw_uid
|
||||
for dirpath, dirnames, filenames in os.walk(path):
|
||||
owner_changed = (os.stat(dirpath).st_uid != uid)
|
||||
if owner_changed is True:
|
||||
changed = owner_changed
|
||||
for dir in [os.path.join(dirpath, d) for d in dirnames]:
|
||||
owner_changed = (os.stat(dir).st_uid != uid)
|
||||
if owner_changed is True:
|
||||
changed = owner_changed
|
||||
for file in [os.path.join(dirpath, f) for f in filenames]:
|
||||
owner_changed = (os.stat(file).st_uid != uid)
|
||||
if owner_changed is True:
|
||||
changed = owner_changed
|
||||
if group is not None:
|
||||
if not module.check_mode:
|
||||
for dirpath, dirnames, filenames in os.walk(path):
|
||||
group_changed = module.set_group_if_different(dirpath, group, False)
|
||||
if group_changed is True:
|
||||
changed = group_changed
|
||||
for dir in [os.path.join(dirpath, d) for d in dirnames]:
|
||||
group_changed = module.set_group_if_different(dir, group, False)
|
||||
if group_changed is True:
|
||||
changed = group_changed
|
||||
for file in [os.path.join(dirpath, f) for f in filenames]:
|
||||
group_changed = module.set_group_if_different(file, group, False)
|
||||
if group_changed is True:
|
||||
changed = group_changed
|
||||
else:
|
||||
gid = grp.getgrnam(group).gr_gid
|
||||
for dirpath, dirnames, filenames in os.walk(path):
|
||||
group_changed = (os.stat(dirpath).st_gid != gid)
|
||||
if group_changed is True:
|
||||
changed = group_changed
|
||||
for dir in [os.path.join(dirpath, d) for d in dirnames]:
|
||||
group_changed = (os.stat(dir).st_gid != gid)
|
||||
if group_changed is True:
|
||||
changed = group_changed
|
||||
for file in [os.path.join(dirpath, f) for f in filenames]:
|
||||
group_changed = (os.stat(file).st_gid != gid)
|
||||
if group_changed is True:
|
||||
changed = group_changed
|
||||
|
||||
return changed
|
||||
|
||||
|
||||
def copy_diff_files(src, dest, module):
|
||||
changed = False
|
||||
owner = module.params['owner']
|
||||
group = module.params['group']
|
||||
local_follow = module.params['local_follow']
|
||||
diff_files = filecmp.dircmp(src, dest).diff_files
|
||||
if len(diff_files):
|
||||
changed = True
|
||||
if not module.check_mode:
|
||||
for item in diff_files:
|
||||
src_item_path = os.path.join(src, item)
|
||||
dest_item_path = os.path.join(dest, item)
|
||||
b_src_item_path = to_bytes(src_item_path, errors='surrogate_or_strict')
|
||||
b_dest_item_path = to_bytes(dest_item_path, errors='surrogate_or_strict')
|
||||
if os.path.islink(b_src_item_path) and local_follow is False:
|
||||
linkto = os.readlink(b_src_item_path)
|
||||
os.symlink(linkto, b_dest_item_path)
|
||||
else:
|
||||
shutil.copyfile(b_src_item_path, b_dest_item_path)
|
||||
shutil.copymode(b_src_item_path, b_dest_item_path)
|
||||
|
||||
if owner is not None:
|
||||
module.set_owner_if_different(b_dest_item_path, owner, False)
|
||||
if group is not None:
|
||||
module.set_group_if_different(b_dest_item_path, group, False)
|
||||
changed = True
|
||||
return changed
|
||||
|
||||
|
||||
def copy_left_only(src, dest, module):
|
||||
changed = False
|
||||
owner = module.params['owner']
|
||||
group = module.params['group']
|
||||
local_follow = module.params['local_follow']
|
||||
left_only = filecmp.dircmp(src, dest).left_only
|
||||
if len(left_only):
|
||||
changed = True
|
||||
if not module.check_mode:
|
||||
for item in left_only:
|
||||
src_item_path = os.path.join(src, item)
|
||||
dest_item_path = os.path.join(dest, item)
|
||||
b_src_item_path = to_bytes(src_item_path, errors='surrogate_or_strict')
|
||||
b_dest_item_path = to_bytes(dest_item_path, errors='surrogate_or_strict')
|
||||
|
||||
if os.path.islink(b_src_item_path) and os.path.isdir(b_src_item_path) and local_follow is True:
|
||||
shutil.copytree(b_src_item_path, b_dest_item_path, symlinks=not(local_follow))
|
||||
chown_recursive(b_dest_item_path, module)
|
||||
|
||||
if os.path.islink(b_src_item_path) and os.path.isdir(b_src_item_path) and local_follow is False:
|
||||
linkto = os.readlink(b_src_item_path)
|
||||
os.symlink(linkto, b_dest_item_path)
|
||||
|
||||
if os.path.islink(b_src_item_path) and os.path.isfile(b_src_item_path) and local_follow is True:
|
||||
shutil.copyfile(b_src_item_path, b_dest_item_path)
|
||||
if owner is not None:
|
||||
module.set_owner_if_different(b_dest_item_path, owner, False)
|
||||
if group is not None:
|
||||
module.set_group_if_different(b_dest_item_path, group, False)
|
||||
|
||||
if os.path.islink(b_src_item_path) and os.path.isfile(b_src_item_path) and local_follow is False:
|
||||
linkto = os.readlink(b_src_item_path)
|
||||
os.symlink(linkto, b_dest_item_path)
|
||||
|
||||
if not os.path.islink(b_src_item_path) and os.path.isfile(b_src_item_path):
|
||||
shutil.copyfile(b_src_item_path, b_dest_item_path)
|
||||
shutil.copymode(b_src_item_path, b_dest_item_path)
|
||||
if owner is not None:
|
||||
module.set_owner_if_different(b_dest_item_path, owner, False)
|
||||
if group is not None:
|
||||
module.set_group_if_different(b_dest_item_path, group, False)
|
||||
|
||||
if not os.path.islink(b_src_item_path) and os.path.isdir(b_src_item_path):
|
||||
shutil.copytree(b_src_item_path, b_dest_item_path, symlinks=not(local_follow))
|
||||
chown_recursive(b_dest_item_path, module)
|
||||
|
||||
changed = True
|
||||
return changed
|
||||
|
||||
|
||||
def copy_common_dirs(src, dest, module):
|
||||
changed = False
|
||||
common_dirs = filecmp.dircmp(src, dest).common_dirs
|
||||
for item in common_dirs:
|
||||
src_item_path = os.path.join(src, item)
|
||||
dest_item_path = os.path.join(dest, item)
|
||||
b_src_item_path = to_bytes(src_item_path, errors='surrogate_or_strict')
|
||||
b_dest_item_path = to_bytes(dest_item_path, errors='surrogate_or_strict')
|
||||
diff_files_changed = copy_diff_files(b_src_item_path, b_dest_item_path, module)
|
||||
left_only_changed = copy_left_only(b_src_item_path, b_dest_item_path, module)
|
||||
if diff_files_changed or left_only_changed:
|
||||
changed = True
|
||||
|
||||
# recurse into subdirectory
|
||||
changed = changed or copy_common_dirs(os.path.join(src, item), os.path.join(dest, item), module)
|
||||
return changed
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
global module
|
||||
|
||||
module = AnsibleModule(
|
||||
# not checking because of daisy chain to file module
|
||||
argument_spec=dict(
|
||||
src=dict(type='path'),
|
||||
_original_basename=dict(type='str'), # used to handle 'dest is a directory' via template, a slight hack
|
||||
content=dict(type='str', no_log=True),
|
||||
dest=dict(type='path', required=True),
|
||||
backup=dict(type='bool', default=False),
|
||||
force=dict(type='bool', default=True, aliases=['thirsty']),
|
||||
validate=dict(type='str'),
|
||||
directory_mode=dict(type='raw'),
|
||||
remote_src=dict(type='bool'),
|
||||
local_follow=dict(type='bool'),
|
||||
checksum=dict(type='str'),
|
||||
),
|
||||
add_file_common_args=True,
|
||||
supports_check_mode=True,
|
||||
)
|
||||
|
||||
if module.params.get('thirsty'):
|
||||
module.deprecate('The alias "thirsty" has been deprecated and will be removed, use "force" instead', version='2.13')
|
||||
|
||||
src = module.params['src']
|
||||
b_src = to_bytes(src, errors='surrogate_or_strict')
|
||||
dest = module.params['dest']
|
||||
# Make sure we always have a directory component for later processing
|
||||
if os.path.sep not in dest:
|
||||
dest = '.{0}{1}'.format(os.path.sep, dest)
|
||||
b_dest = to_bytes(dest, errors='surrogate_or_strict')
|
||||
backup = module.params['backup']
|
||||
force = module.params['force']
|
||||
_original_basename = module.params.get('_original_basename', None)
|
||||
validate = module.params.get('validate', None)
|
||||
follow = module.params['follow']
|
||||
local_follow = module.params['local_follow']
|
||||
mode = module.params['mode']
|
||||
owner = module.params['owner']
|
||||
group = module.params['group']
|
||||
remote_src = module.params['remote_src']
|
||||
checksum = module.params['checksum']
|
||||
|
||||
if not os.path.exists(b_src):
|
||||
module.fail_json(msg="Source %s not found" % (src))
|
||||
if not os.access(b_src, os.R_OK):
|
||||
module.fail_json(msg="Source %s not readable" % (src))
|
||||
|
||||
# Preserve is usually handled in the action plugin but mode + remote_src has to be done on the
|
||||
# remote host
|
||||
if module.params['mode'] == 'preserve':
|
||||
module.params['mode'] = '0%03o' % stat.S_IMODE(os.stat(b_src).st_mode)
|
||||
mode = module.params['mode']
|
||||
|
||||
checksum_dest = None
|
||||
|
||||
if os.path.isfile(src):
|
||||
checksum_src = module.sha1(src)
|
||||
else:
|
||||
checksum_src = None
|
||||
|
||||
# Backwards compat only. This will be None in FIPS mode
|
||||
try:
|
||||
if os.path.isfile(src):
|
||||
md5sum_src = module.md5(src)
|
||||
else:
|
||||
md5sum_src = None
|
||||
except ValueError:
|
||||
md5sum_src = None
|
||||
|
||||
changed = False
|
||||
|
||||
if checksum and checksum_src != checksum:
|
||||
module.fail_json(
|
||||
msg='Copied file does not match the expected checksum. Transfer failed.',
|
||||
checksum=checksum_src,
|
||||
expected_checksum=checksum
|
||||
)
|
||||
|
||||
# Special handling for recursive copy - create intermediate dirs
|
||||
if dest.endswith(os.sep):
|
||||
if _original_basename:
|
||||
dest = os.path.join(dest, _original_basename)
|
||||
b_dest = to_bytes(dest, errors='surrogate_or_strict')
|
||||
dirname = os.path.dirname(dest)
|
||||
b_dirname = to_bytes(dirname, errors='surrogate_or_strict')
|
||||
if not os.path.exists(b_dirname):
|
||||
try:
|
||||
(pre_existing_dir, new_directory_list) = split_pre_existing_dir(dirname)
|
||||
except AnsibleModuleError as e:
|
||||
e.result['msg'] += ' Could not copy to {0}'.format(dest)
|
||||
module.fail_json(**e.results)
|
||||
|
||||
os.makedirs(b_dirname)
|
||||
directory_args = module.load_file_common_arguments(module.params)
|
||||
directory_mode = module.params["directory_mode"]
|
||||
if directory_mode is not None:
|
||||
directory_args['mode'] = directory_mode
|
||||
else:
|
||||
directory_args['mode'] = None
|
||||
adjust_recursive_directory_permissions(pre_existing_dir, new_directory_list, module, directory_args, changed)
|
||||
|
||||
if os.path.isdir(b_dest):
|
||||
basename = os.path.basename(src)
|
||||
if _original_basename:
|
||||
basename = _original_basename
|
||||
dest = os.path.join(dest, basename)
|
||||
b_dest = to_bytes(dest, errors='surrogate_or_strict')
|
||||
|
||||
if os.path.exists(b_dest):
|
||||
if os.path.islink(b_dest) and follow:
|
||||
b_dest = os.path.realpath(b_dest)
|
||||
dest = to_native(b_dest, errors='surrogate_or_strict')
|
||||
if not force:
|
||||
module.exit_json(msg="file already exists", src=src, dest=dest, changed=False)
|
||||
if os.access(b_dest, os.R_OK) and os.path.isfile(b_dest):
|
||||
checksum_dest = module.sha1(dest)
|
||||
else:
|
||||
if not os.path.exists(os.path.dirname(b_dest)):
|
||||
try:
|
||||
# os.path.exists() can return false in some
|
||||
# circumstances where the directory does not have
|
||||
# the execute bit for the current user set, in
|
||||
# which case the stat() call will raise an OSError
|
||||
os.stat(os.path.dirname(b_dest))
|
||||
except OSError as e:
|
||||
if "permission denied" in to_native(e).lower():
|
||||
module.fail_json(msg="Destination directory %s is not accessible" % (os.path.dirname(dest)))
|
||||
module.fail_json(msg="Destination directory %s does not exist" % (os.path.dirname(dest)))
|
||||
|
||||
if not os.access(os.path.dirname(b_dest), os.W_OK) and not module.params['unsafe_writes']:
|
||||
module.fail_json(msg="Destination %s not writable" % (os.path.dirname(dest)))
|
||||
|
||||
backup_file = None
|
||||
if checksum_src != checksum_dest or os.path.islink(b_dest):
|
||||
if not module.check_mode:
|
||||
try:
|
||||
if backup:
|
||||
if os.path.exists(b_dest):
|
||||
backup_file = module.backup_local(dest)
|
||||
# allow for conversion from symlink.
|
||||
if os.path.islink(b_dest):
|
||||
os.unlink(b_dest)
|
||||
open(b_dest, 'w').close()
|
||||
if validate:
|
||||
# if we have a mode, make sure we set it on the temporary
|
||||
# file source as some validations may require it
|
||||
if mode is not None:
|
||||
module.set_mode_if_different(src, mode, False)
|
||||
if owner is not None:
|
||||
module.set_owner_if_different(src, owner, False)
|
||||
if group is not None:
|
||||
module.set_group_if_different(src, group, False)
|
||||
if "%s" not in validate:
|
||||
module.fail_json(msg="validate must contain %%s: %s" % (validate))
|
||||
(rc, out, err) = module.run_command(validate % src)
|
||||
if rc != 0:
|
||||
module.fail_json(msg="failed to validate", exit_status=rc, stdout=out, stderr=err)
|
||||
b_mysrc = b_src
|
||||
if remote_src and os.path.isfile(b_src):
|
||||
_, b_mysrc = tempfile.mkstemp(dir=os.path.dirname(b_dest))
|
||||
|
||||
shutil.copyfile(b_src, b_mysrc)
|
||||
try:
|
||||
shutil.copystat(b_src, b_mysrc)
|
||||
except OSError as err:
|
||||
if err.errno == errno.ENOSYS and mode == "preserve":
|
||||
module.warn("Unable to copy stats {0}".format(to_native(b_src)))
|
||||
else:
|
||||
raise
|
||||
|
||||
# might be needed below
|
||||
if PY3 and hasattr(os, 'listxattr'):
|
||||
try:
|
||||
src_has_acls = 'system.posix_acl_access' in os.listxattr(src)
|
||||
except Exception as e:
|
||||
# assume unwanted ACLs by default
|
||||
src_has_acls = True
|
||||
|
||||
module.atomic_move(b_mysrc, dest, unsafe_writes=module.params['unsafe_writes'])
|
||||
|
||||
if PY3 and hasattr(os, 'listxattr') and platform.system() == 'Linux' and not remote_src:
|
||||
# atomic_move used above to copy src into dest might, in some cases,
|
||||
# use shutil.copy2 which in turn uses shutil.copystat.
|
||||
# Since Python 3.3, shutil.copystat copies file extended attributes:
|
||||
# https://docs.python.org/3/library/shutil.html#shutil.copystat
|
||||
# os.listxattr (along with others) was added to handle the operation.
|
||||
|
||||
# This means that on Python 3 we are copying the extended attributes which includes
|
||||
# the ACLs on some systems - further limited to Linux as the documentation above claims
|
||||
# that the extended attributes are copied only on Linux. Also, os.listxattr is only
|
||||
# available on Linux.
|
||||
|
||||
# If not remote_src, then the file was copied from the controller. In that
|
||||
# case, any filesystem ACLs are artifacts of the copy rather than preservation
|
||||
# of existing attributes. Get rid of them:
|
||||
|
||||
if src_has_acls:
|
||||
# FIXME If dest has any default ACLs, there are not applied to src now because
|
||||
# they were overridden by copystat. Should/can we do anything about this?
|
||||
# 'system.posix_acl_default' in os.listxattr(os.path.dirname(b_dest))
|
||||
|
||||
try:
|
||||
clear_facls(dest)
|
||||
except ValueError as e:
|
||||
if 'setfacl' in to_native(e):
|
||||
# No setfacl so we're okay. The controller couldn't have set a facl
|
||||
# without the setfacl command
|
||||
pass
|
||||
else:
|
||||
raise
|
||||
except RuntimeError as e:
|
||||
# setfacl failed.
|
||||
if 'Operation not supported' in to_native(e):
|
||||
# The file system does not support ACLs.
|
||||
pass
|
||||
else:
|
||||
raise
|
||||
|
||||
except (IOError, OSError):
|
||||
module.fail_json(msg="failed to copy: %s to %s" % (src, dest), traceback=traceback.format_exc())
|
||||
changed = True
|
||||
else:
|
||||
changed = False
|
||||
|
||||
if checksum_src is None and checksum_dest is None:
|
||||
if remote_src and os.path.isdir(module.params['src']):
|
||||
b_src = to_bytes(module.params['src'], errors='surrogate_or_strict')
|
||||
b_dest = to_bytes(module.params['dest'], errors='surrogate_or_strict')
|
||||
|
||||
if src.endswith(os.path.sep) and os.path.isdir(module.params['dest']):
|
||||
diff_files_changed = copy_diff_files(b_src, b_dest, module)
|
||||
left_only_changed = copy_left_only(b_src, b_dest, module)
|
||||
common_dirs_changed = copy_common_dirs(b_src, b_dest, module)
|
||||
owner_group_changed = chown_recursive(b_dest, module)
|
||||
if diff_files_changed or left_only_changed or common_dirs_changed or owner_group_changed:
|
||||
changed = True
|
||||
|
||||
if src.endswith(os.path.sep) and not os.path.exists(module.params['dest']):
|
||||
b_basename = to_bytes(os.path.basename(src), errors='surrogate_or_strict')
|
||||
b_dest = to_bytes(os.path.join(b_dest, b_basename), errors='surrogate_or_strict')
|
||||
b_src = to_bytes(os.path.join(module.params['src'], ""), errors='surrogate_or_strict')
|
||||
if not module.check_mode:
|
||||
shutil.copytree(b_src, b_dest, symlinks=not(local_follow))
|
||||
chown_recursive(dest, module)
|
||||
changed = True
|
||||
|
||||
if not src.endswith(os.path.sep) and os.path.isdir(module.params['dest']):
|
||||
b_basename = to_bytes(os.path.basename(src), errors='surrogate_or_strict')
|
||||
b_dest = to_bytes(os.path.join(b_dest, b_basename), errors='surrogate_or_strict')
|
||||
b_src = to_bytes(os.path.join(module.params['src'], ""), errors='surrogate_or_strict')
|
||||
if not module.check_mode and not os.path.exists(b_dest):
|
||||
shutil.copytree(b_src, b_dest, symlinks=not(local_follow))
|
||||
changed = True
|
||||
chown_recursive(dest, module)
|
||||
if module.check_mode and not os.path.exists(b_dest):
|
||||
changed = True
|
||||
if os.path.exists(b_dest):
|
||||
diff_files_changed = copy_diff_files(b_src, b_dest, module)
|
||||
left_only_changed = copy_left_only(b_src, b_dest, module)
|
||||
common_dirs_changed = copy_common_dirs(b_src, b_dest, module)
|
||||
owner_group_changed = chown_recursive(b_dest, module)
|
||||
if diff_files_changed or left_only_changed or common_dirs_changed or owner_group_changed:
|
||||
changed = True
|
||||
|
||||
if not src.endswith(os.path.sep) and not os.path.exists(module.params['dest']):
|
||||
b_basename = to_bytes(os.path.basename(module.params['src']), errors='surrogate_or_strict')
|
||||
b_dest = to_bytes(os.path.join(b_dest, b_basename), errors='surrogate_or_strict')
|
||||
if not module.check_mode and not os.path.exists(b_dest):
|
||||
os.makedirs(b_dest)
|
||||
b_src = to_bytes(os.path.join(module.params['src'], ""), errors='surrogate_or_strict')
|
||||
diff_files_changed = copy_diff_files(b_src, b_dest, module)
|
||||
left_only_changed = copy_left_only(b_src, b_dest, module)
|
||||
common_dirs_changed = copy_common_dirs(b_src, b_dest, module)
|
||||
owner_group_changed = chown_recursive(b_dest, module)
|
||||
if diff_files_changed or left_only_changed or common_dirs_changed or owner_group_changed:
|
||||
changed = True
|
||||
if module.check_mode and not os.path.exists(b_dest):
|
||||
changed = True
|
||||
|
||||
res_args = dict(
|
||||
dest=dest, src=src, md5sum=md5sum_src, checksum=checksum_src, changed=changed
|
||||
)
|
||||
if backup_file:
|
||||
res_args['backup_file'] = backup_file
|
||||
|
||||
module.params['dest'] = dest
|
||||
if not module.check_mode:
|
||||
file_args = module.load_file_common_arguments(module.params)
|
||||
res_args['changed'] = module.set_fs_attributes_if_different(file_args, res_args['changed'])
|
||||
|
||||
module.exit_json(**res_args)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -1,142 +0,0 @@
|
||||
- name: "htcondor docker image"
|
||||
file:
|
||||
path: "/container/docker-images/htcondor"
|
||||
state: directory
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
mode: "u=rwx,g=rwx,o=rx"
|
||||
|
||||
- copy:
|
||||
dest: "/container/docker-images/htcondor/Dockerfile"
|
||||
src: "htcondor.Dockerfile"
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
register: cp_dockerfile
|
||||
|
||||
- docker_image:
|
||||
name: "htcondor"
|
||||
# pull: False
|
||||
build:
|
||||
pull: False
|
||||
path: "/container/docker-images/htcondor"
|
||||
source: build
|
||||
force_source: "{{cp_dockerfile.changed}}"
|
||||
|
||||
- name: "copy htcondor container configuration"
|
||||
copy:
|
||||
src: "conf/{{item}}/"
|
||||
dest: "/container/volumes/{{item}}/"
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
mode: "u=rwx,g=rwx"
|
||||
with_items: [ "cm", "exec", "sub", "common"]
|
||||
|
||||
- name: "check if pool shared secret exists"
|
||||
stat:
|
||||
path: "/container/volumes/common/passwords.d/POOL"
|
||||
register: pool_pw
|
||||
|
||||
- block:
|
||||
- name: "create temporary password store"
|
||||
tempfile:
|
||||
state: directory
|
||||
register: pool_pw_tmp
|
||||
|
||||
- name: "generate pool password"
|
||||
copy:
|
||||
dest: "{{pool_pw_tmp.path}}/poolpw"
|
||||
content: "{{lookup('password','/dev/null')}}"
|
||||
no_log: True
|
||||
|
||||
- name: "install pool password"
|
||||
docker_container:
|
||||
name: "condor-common"
|
||||
image: htcondor
|
||||
state: started
|
||||
volumes:
|
||||
- "/container/volumes/common/:/etc/condor/:rw"
|
||||
- "{{pool_pw_tmp.path}}:/tmp/poolpw:ro"
|
||||
detach: False
|
||||
cleanup: True
|
||||
command: "condor_store_cred add -c -i /tmp/poolpw/poolpw"
|
||||
|
||||
- name: "remove tokens since pool password (cert) changed"
|
||||
file:
|
||||
path: "/container/volumes/{{item}}/tokens.d/condor@htc.local"
|
||||
state: absent
|
||||
with_items: [ "cm", "exec", "sub" ]
|
||||
|
||||
always:
|
||||
- name: "remove temporary password store"
|
||||
file:
|
||||
path: "{{pool_pw_tmp.path}}"
|
||||
state: absent
|
||||
when: pool_pw_tmp is defined and pool_pw_tmp.path
|
||||
when: not pool_pw.stat.exists
|
||||
|
||||
- name: "sync common files to individual containers"
|
||||
copy_2:
|
||||
remote_src: True
|
||||
force: True
|
||||
directory_mode: preserve
|
||||
mode: preserve
|
||||
src: "/container/volumes/common/"
|
||||
dest: "/container/volumes/{{item}}/"
|
||||
with_items: [ "cm", "exec", "sub"]
|
||||
|
||||
|
||||
- name: "collect tokens to generate"
|
||||
stat:
|
||||
path: "/container/volumes/{{item}}/tokens.d/condor@htc.local"
|
||||
with_items: [ "cm", "exec", "sub" ]
|
||||
register: tokens_state
|
||||
|
||||
- name: "generate tokens"
|
||||
docker_container:
|
||||
name: "condor-common"
|
||||
image: htcondor
|
||||
state: started
|
||||
volumes:
|
||||
- "/container/volumes/{{item}}/:/etc/condor/:rw"
|
||||
detach: False
|
||||
cleanup: True
|
||||
command: "condor_token_create -identity condor@$(domainname) -token /etc/condor/tokens.d/condor@htc.local"
|
||||
with_items: "{{tokens_state.results | rejectattr('stat.exists') | map(attribute='item') | list }}"
|
||||
|
||||
|
||||
- name: "create docker network to make service discovery work"
|
||||
docker_network:
|
||||
name: condor
|
||||
state: present
|
||||
|
||||
# TODO: reserve some address using docker_network_info and assign as aux
|
||||
# address to enable cm to get a static address in order to be reachable from
|
||||
# htcondor running on docker host to enable submitting jobs.
|
||||
|
||||
- name: "run htcondor containers"
|
||||
docker_container:
|
||||
name: "condor-{{item}}"
|
||||
hostname: "condor-{{item}}"
|
||||
domainname: "htc.local"
|
||||
image: htcondor
|
||||
state: started
|
||||
detach: True
|
||||
cleanup: True
|
||||
networks_cli_compatible: True
|
||||
networks:
|
||||
- name: "condor"
|
||||
aliases: [ "condor-{{item}}.htc.local" ]
|
||||
volumes:
|
||||
- "/container/volumes/{{item}}/:/etc/condor/:rw"
|
||||
with_items: [ "cm", "exec", "sub"]
|
||||
# auto_remove: True
|
||||
# mounts:
|
||||
# src: /container/volumes/cm/
|
||||
# dest: /etc/condor/
|
||||
|
||||
|
||||
#- add_host:
|
||||
# hostname: foo
|
||||
# ansible_connection: docker_api
|
||||
# docker_host: ssh://ed-c7-1.virt.magni.thoto.net
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
- name: "install docker yum repository"
|
||||
get_url:
|
||||
url: "https://download.docker.com/linux/centos/docker-ce.repo"
|
||||
dest: "/etc/yum.repos.d/docker-ce.repo"
|
||||
checksum: sha256:8ab5599eef0afcac10cbd3e8670873efee20fcceb5fb3526a62edeade603cec7
|
||||
|
||||
- yum:
|
||||
name: [ "docker-ce", "python-docker-py" ] # latter for ansible modules
|
||||
state: present
|
||||
|
||||
- name: "partition container image disk"
|
||||
parted:
|
||||
device: /dev/vdb
|
||||
number: 1
|
||||
state: present
|
||||
# fs_type: xfs
|
||||
|
||||
- filesystem:
|
||||
dev: /dev/vdb1
|
||||
fstype: xfs
|
||||
opts: "-L image-store"
|
||||
|
||||
- mount:
|
||||
path: "/container"
|
||||
src: "/dev/vdb1"
|
||||
fstype: xfs
|
||||
opts: "noatime"
|
||||
state: mounted
|
||||
|
||||
- file:
|
||||
path: "/container/docker"
|
||||
state: directory
|
||||
owner: root
|
||||
group: root
|
||||
mode: "u=rwx,g=x,o=x"
|
||||
|
||||
- name: "link docker configuration to new container partition"
|
||||
file:
|
||||
path: "/var/lib/docker"
|
||||
src: "/container/docker"
|
||||
state: link
|
||||
owner: root
|
||||
group: root
|
||||
mode: "u=rwx,g=x,o=x"
|
||||
|
||||
- name: "enable docker service in systemd"
|
||||
service:
|
||||
name: docker
|
||||
enabled: True
|
||||
state: started
|
||||
|
||||
- name: "fix user privileges to run docker"
|
||||
user:
|
||||
name: "{{unpriv_user}}"
|
||||
groups: docker
|
||||
append: True
|
||||
@@ -1,18 +0,0 @@
|
||||
- yum:
|
||||
name: singularity
|
||||
state: present
|
||||
|
||||
- name: "enable user namespaces for singularity fakeroot function"
|
||||
sysctl:
|
||||
name: user.max_user_namespaces
|
||||
value: "15000"
|
||||
sysctl_file: /etc/sysctl.d/90-max_net_namespaces.conf
|
||||
|
||||
- name: "enable user {{unpriv_user}} for fakeroot access"
|
||||
lineinfile:
|
||||
line: "{{unpriv_user}}:4294836224:65536"
|
||||
dest: "{{item}}"
|
||||
with_items: ["/etc/subuid", "/etc/subgid"]
|
||||
# the task above idempotentially does the same as:
|
||||
#- shell:
|
||||
# command: singularity config fakeroot --add thoto
|
||||
@@ -1,9 +0,0 @@
|
||||
slurm_user: slurm
|
||||
slurm_log_path_ctld: /var/log/slurm/slurmctld.log
|
||||
slurm_log_path_d: /var/log/slurm/slurmd.log
|
||||
slurm_log_path_sched: /var/log/slurm/slurmsched.log
|
||||
slurm_prefix: slurm
|
||||
slurm_domain: slurm.local
|
||||
container_privileged: False
|
||||
docker_network: slurm
|
||||
slurm_image_prefix: slurm
|
||||
@@ -1,18 +0,0 @@
|
||||
#!/bin/bash
|
||||
function trp_term(){
|
||||
echo pkill -P $pids
|
||||
for j in $pids ; do
|
||||
pkill -P $j
|
||||
kill -SIGTERM $j
|
||||
done
|
||||
}
|
||||
trap trp_term SIGINT SIGTERM
|
||||
pids=""
|
||||
for i in /etc/docker-init.d/* ; do
|
||||
[ ! -f $i ] && break
|
||||
$i &
|
||||
pids="$pids $!"
|
||||
done
|
||||
wait $pids
|
||||
|
||||
# TODO: call start scripts like "foo.sh start" and "foo.sh stop" to avoid pkill
|
||||
@@ -1,7 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
if [ -f "/etc/munge/munge.key" ] ; then
|
||||
chown munge:munge /etc/munge/munge.key
|
||||
chmod 600 /etc/munge/munge.key
|
||||
fi
|
||||
@@ -1,8 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
for i in /usr/local/lib/entrypoints.d/* ; do
|
||||
[ -f $i ] && /bin/sh $i || break
|
||||
done
|
||||
|
||||
exec "${@:-/bin/bash}"
|
||||
@@ -1,35 +0,0 @@
|
||||
FROM docker.io/library/centos:7
|
||||
|
||||
RUN yum install -y epel-release && \
|
||||
yum install -y slurm && \
|
||||
yum clean all && rm -rf /var/cache/yum
|
||||
|
||||
RUN yum install -y less iproute bind-utils nmap-ncat net-tools && \
|
||||
yum clean all && rm -rf /var/cache/yum
|
||||
|
||||
RUN mkdir -p /usr/local/lib/entrypoints.d/
|
||||
|
||||
COPY --chown=root:root entry-munge.sh /usr/local/lib/entrypoints.d/10-munge.sh
|
||||
COPY --chown=root:root entrypoint.sh /usr/local/sbin/entrypoint.sh
|
||||
|
||||
RUN chmod 755 /usr/local/lib/entrypoints.d/10-munge.sh && \
|
||||
chmod 755 /usr/local/sbin/entrypoint.sh
|
||||
|
||||
ENTRYPOINT [ "/usr/local/sbin/entrypoint.sh" ]
|
||||
|
||||
ARG slurmuser=slurm
|
||||
ENV slurmuser=${slurmuser}
|
||||
|
||||
RUN useradd -d /var/lib/slurm -m --no-log-init --system $slurmuser &&\
|
||||
slurm-setuser -u $slurmuser -g $slurmuser -y
|
||||
|
||||
COPY docker-init /usr/local/sbin/docker-init
|
||||
RUN mkdir /etc/docker-init.d && chmod 755 /usr/local/sbin/docker-init
|
||||
COPY start-scripts/10-munge /etc/docker-init.d/10-munge
|
||||
RUN chmod 755 /etc/docker-init.d/10-munge
|
||||
|
||||
ARG moreusers
|
||||
RUN function mu { [ -z "$1" ] || useradd -d $2 -m --no-log-init --system $1 ;};\
|
||||
echo "${moreusers}" | tr ',' '\n' | while read i ; do mu $i ; done
|
||||
|
||||
CMD /usr/local/sbin/docker-init
|
||||
@@ -1,11 +0,0 @@
|
||||
FROM slurm:base
|
||||
|
||||
RUN yum install -y slurm-slurmctld && \
|
||||
yum clean all && rm -rf /var/cache/yum
|
||||
|
||||
COPY start-scripts/20-slurmctld /etc/docker-init.d/20-slurmctld
|
||||
RUN chmod 755 /etc/docker-init.d/20-slurmctld
|
||||
|
||||
ENV SLURMCTLD_LOG_PATH="/var/log/slurm/slurmctld.log"
|
||||
ENV SLURMD_LOG_PATH="/var/log/slurm/slurmd.log"
|
||||
ENV SLURM_SCHED_LOG_PATH="/var/log/slurm/slurmsched.log"
|
||||
@@ -1,14 +0,0 @@
|
||||
FROM slurm:base
|
||||
|
||||
RUN yum install -y slurm-slurmd && \
|
||||
yum clean all && rm -rf /var/cache/yum
|
||||
|
||||
COPY start-scripts/30-slurmd /etc/docker-init.d/30-slurmd
|
||||
RUN chmod 755 /etc/docker-init.d/30-slurmd
|
||||
|
||||
ENV SLURMCTLD_LOG_PATH="/var/log/slurm/slurmctld.log"
|
||||
ENV SLURMD_LOG_PATH="/var/log/slurm/slurmd.log"
|
||||
ENV SLURM_SCHED_LOG_PATH="/var/log/slurm/slurmsched.log"
|
||||
|
||||
RUN yum install -y singularity && \
|
||||
yum clean all && rm -rf /var/cache/yum
|
||||
@@ -1,2 +0,0 @@
|
||||
#!/bin/sh
|
||||
exec su -s /bin/sh -c "munged -F" munge
|
||||
@@ -1,4 +0,0 @@
|
||||
#!/bin/sh
|
||||
su -s /bin/sh -c "slurmctld -D" ${slurmuser} 2>/dev/null 1>/dev/null &
|
||||
tail --retry --pid $! -f ${SLURMCTLD_LOG_PATH} ${SLURM_SCHED_LOG_PATH}
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
#!/bin/sh
|
||||
slurmd -D 2>/dev/null 1>/dev/null &
|
||||
tail --retry --pid $! -f ${SLURMD_LOG_PATH} ${SLURM_SCHED_LOG_PATH}
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
- name: reconfigure slurm
|
||||
shell:
|
||||
cmd: "docker container exec -it {{slurm_prefix}}-ctl scontrol reconfigure || docker container restart {{slurm_prefix}}-ctl && docker container exec -it {{slurm_prefix}}-ctl scontrol reconfigure"
|
||||
@@ -1,29 +0,0 @@
|
||||
- name: build slurm base docker image
|
||||
include_tasks: dockerimage_build.yml
|
||||
vars:
|
||||
slurm_image_prefix: "{{slurm_base_image_prefix | default('slurm') }}"
|
||||
image_name: base
|
||||
dockerfile: "{{lookup('file', 'slurm-base.Dockerfile')}}"
|
||||
files:
|
||||
- dest: entrypoint.sh
|
||||
content: "{{ lookup('file', 'entrypoint.sh') }}"
|
||||
- dest: entry-munge.sh
|
||||
content: "{{ lookup('file', 'entry-munge.sh') }}"
|
||||
- dest: docker-init
|
||||
content: "{{ lookup('file', 'docker-init') }}"
|
||||
- dest: start-scripts/10-munge
|
||||
content: "{{ lookup('file', 'start-scripts/10-munge') }}"
|
||||
image_args:
|
||||
moreusers: >-
|
||||
{% for a in slurm_user_accounts | default([]) -%}
|
||||
{{a['name']}} {{a['dir']}}{{loop.last | ternary('',',')}}
|
||||
{%- endfor %}
|
||||
when: not slurm_baseimg_build_chg | default(False)
|
||||
|
||||
- set_fact:
|
||||
slurm_baseimg_build_chg:
|
||||
"{{(slurm_baseimg_build_chg | default(False)) or
|
||||
slurm_img_build.changed}}"
|
||||
|
||||
- name: "build slurm base docker image {{image_name}}"
|
||||
include_tasks: dockerimage_build.yml
|
||||
@@ -1,43 +0,0 @@
|
||||
- name: create directories for docker image build
|
||||
file:
|
||||
path: "/container/docker-images/{{slurm_image_prefix}}-{{image_name}}/{{item}}"
|
||||
state: directory
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
loop: "{{ [''] + (files | map(attribute='dest') | map('dirname') |
|
||||
unique | select | list) }}"
|
||||
|
||||
- name: "copy Dockerfile {{slurm_image_prefix}}:{{image_name}}"
|
||||
copy:
|
||||
content: "{{dockerfile}}"
|
||||
dest: "/container/docker-images/{{slurm_image_prefix}}-{{image_name}}/Dockerfile"
|
||||
owner: "{{unpriv_user}}"
|
||||
group: docker
|
||||
register: slurm_cp_dockerfile
|
||||
|
||||
- name: copy requisite files
|
||||
copy:
|
||||
content: "{{ item.content }}"
|
||||
dest: "/container/docker-images/{{slurm_image_prefix}}-{{image_name}}/{{item.dest}}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: u=rwx,g=rx,o=rx
|
||||
loop: "{{ files | default([]) }}"
|
||||
loop_control:
|
||||
label: "{{ item.dest }}"
|
||||
register: slurm_cp_files
|
||||
|
||||
- name: "build docker image {{slurm_image_prefix}}:{{image_name}}"
|
||||
docker_image:
|
||||
name: "{{slurm_image_prefix}}"
|
||||
tag: "{{image_name}}"
|
||||
# pull: False
|
||||
build:
|
||||
args: "{{image_args | default(omit)}}"
|
||||
pull: False
|
||||
path: "/container/docker-images/{{slurm_image_prefix}}-{{image_name}}/"
|
||||
source: build
|
||||
force_source: "{{slurm_cp_dockerfile.changed or
|
||||
slurm_cp_files.changed or
|
||||
slurm_baseimg_build_chg | default(False) }}"
|
||||
register: slurm_img_build
|
||||
@@ -1,35 +0,0 @@
|
||||
# TODO: this does not work quite right since slurm-ctl does not reach the host
|
||||
# system. sinfo, scontrol etc. work but srun does not!
|
||||
|
||||
- name: "get addresses from docker network"
|
||||
docker_network_info:
|
||||
name: "{{ docker_network }}"
|
||||
register: slurm_network_data
|
||||
|
||||
- name: link host slurm config
|
||||
file:
|
||||
path: "/etc/slurm/slurm.conf"
|
||||
src: "/container/volumes/slurm/slurm.conf"
|
||||
force: True
|
||||
state: link
|
||||
backup: True
|
||||
|
||||
- name: create slurm user
|
||||
user:
|
||||
name: slurm
|
||||
system: True
|
||||
|
||||
- name: place entry of slurm-ctl in host /etc/hosts
|
||||
lineinfile:
|
||||
line: "{{slurm_network_data.network.Containers | dict2items
|
||||
| json_query('[?value.Name==`slurm-ctl`].value.IPv4Address') | first
|
||||
| ipaddr('address') }}\tslurm-ctl"
|
||||
regexp: "^(\\S*)(\\s*)slurm-ctl$"
|
||||
path: /etc/hosts
|
||||
backup: True
|
||||
|
||||
- name: start munge locally
|
||||
service:
|
||||
name: munge
|
||||
enabled: True
|
||||
state: started
|
||||
@@ -1,138 +0,0 @@
|
||||
- name: 'install slurm tools on host'
|
||||
yum:
|
||||
name: [ slurm, slurm-doc ]
|
||||
state: present
|
||||
|
||||
- name: build docker images for slurm
|
||||
include_tasks:
|
||||
file: dockerimage.yml
|
||||
loop:
|
||||
- name: slurmctld
|
||||
dockerfile: "{{ lookup('file', 'slurmctld.Dockerfile') }}"
|
||||
files:
|
||||
- dest: start-scripts/20-slurmctld
|
||||
content: "{{ lookup('file', 'start-scripts/20-slurmctld') }}"
|
||||
- name: slurmd
|
||||
dockerfile: "{{ lookup('file', 'slurmd.Dockerfile') }}"
|
||||
files:
|
||||
- dest: start-scripts/30-slurmd
|
||||
content: "{{ lookup('file', 'start-scripts/30-slurmd') }}"
|
||||
vars:
|
||||
image_name: "{{image.name | default(omit) }}"
|
||||
dockerfile: "{{image.dockerfile | default(omit) }}"
|
||||
files: "{{image.files | default(omit) }}"
|
||||
loop_control:
|
||||
loop_var: image
|
||||
label: "{{ image.name }}"
|
||||
|
||||
- name: generate munge key
|
||||
shell:
|
||||
cmd: dd if=/dev/urandom bs=1 count=1024 >/etc/munge/munge.key
|
||||
creates: /etc/munge/munge.key
|
||||
|
||||
- name: set permissions on munge key
|
||||
file:
|
||||
path: /etc/munge/munge.key
|
||||
owner: munge
|
||||
group: munge
|
||||
mode: u=rw,g=,o=
|
||||
|
||||
- name: create munge key directory for containers
|
||||
file:
|
||||
path: /container/volumes/munge
|
||||
state: directory
|
||||
owner: munge
|
||||
group: munge
|
||||
mode: u=rwx,g=,o=
|
||||
|
||||
- name: copy munge key to docker containers
|
||||
copy:
|
||||
remote_src: True
|
||||
force: true
|
||||
mode: preserve
|
||||
src: /etc/munge/munge.key
|
||||
dest: /container/volumes/munge/munge.key
|
||||
|
||||
- name: make slurm directory
|
||||
file:
|
||||
path: /container/volumes/slurm/
|
||||
state: directory
|
||||
|
||||
- name: "create docker network to make service discovery work"
|
||||
docker_network:
|
||||
name: "{{ docker_network }}"
|
||||
state: present
|
||||
register: slurm_network_data
|
||||
tags: slurm-config
|
||||
|
||||
- name: upload slurm config
|
||||
template:
|
||||
force: true
|
||||
src: "{{item}}.j2"
|
||||
dest: "/container/volumes/slurm/{{item}}"
|
||||
loop:
|
||||
- slurm.conf
|
||||
- cgroup.conf
|
||||
vars:
|
||||
slurm_exec_node_cores: 3
|
||||
slurm_exec_node_mem: 5000 # RealMemory=5964
|
||||
slurm_alloc_nodes_default:
|
||||
- name: "{{slurm_prefix+'-submit1'}}"
|
||||
- name: "{{ inventory_hostname }}"
|
||||
addr: "{{ slurm_network_data.network.IPAM.Config[0].Gateway }}"
|
||||
alloc_nodes: "{{ slurm_alloc_nodes_default + extra_nodes | default([])}}"
|
||||
partitions:
|
||||
- name: cobald
|
||||
nodeprefix: drone
|
||||
num_nodes: 10
|
||||
node_cores: 3
|
||||
node_mem: 4900
|
||||
port: 16818
|
||||
initstate: FUTURE
|
||||
notify: reconfigure slurm
|
||||
tags: slurm-config
|
||||
|
||||
- name: "create docker volume for shared access between nodes"
|
||||
docker_volume:
|
||||
name: slurm-shared
|
||||
state: present
|
||||
|
||||
- name: run slurm docker containers
|
||||
docker_container:
|
||||
name: "{{ slurm_prefix }}-{{ item.machine }}"
|
||||
hostname: "{{ slurm_prefix }}-{{ item.machine }}"
|
||||
domainname: "{{ slurm_domain }}"
|
||||
volumes: "{{ slurm_default_mounts + ( item.extra_mounts | default([]) ) }}"
|
||||
ports: "{{ item.exposed_ports | default([]) }}"
|
||||
networks:
|
||||
- name: "{{ docker_network }}"
|
||||
aliases: "{{ item.aliases | default(omit) }}"
|
||||
env:
|
||||
slurmuser: "{{ slurm_user }}"
|
||||
image: "{{ item.image }}"
|
||||
state: started
|
||||
detach: True
|
||||
cleanup: True
|
||||
privileged: "{{ container_privileged | bool }}"
|
||||
networks_cli_compatible: True
|
||||
interactive: True
|
||||
vars: # see vars/main.yml
|
||||
slurm_nodes_all: "{{ slurm_nodes_exec + slurm_nodes_std }}"
|
||||
loop: "{{ slurm_nodes_all }}"
|
||||
loop_control:
|
||||
label: "{{slurm_prefix}}-{{ item.machine }}"
|
||||
tags: slurm-config
|
||||
|
||||
- name: configure host system to integrate into slurm cluster
|
||||
import_tasks: host-config.yml
|
||||
when: slurm_hostsystem_cluster_access | default(False)
|
||||
|
||||
- name: export facts about slurm cluster to be used by other modules
|
||||
set_fact:
|
||||
slurm:
|
||||
user: "{{slurm_user}}"
|
||||
domain: "{{slurm_domain}}"
|
||||
base_image: "slurm:base"
|
||||
mounts: "{{slurm_default_mounts}}"
|
||||
network: "{{docker_network}}"
|
||||
tags: always
|
||||
@@ -1,11 +0,0 @@
|
||||
###
|
||||
#
|
||||
# Slurm cgroup support configuration file
|
||||
#
|
||||
# See man slurm.conf and man cgroup.conf for further
|
||||
# information on cgroup configuration parameters
|
||||
#--
|
||||
CgroupAutomount=yes
|
||||
|
||||
ConstrainCores=no
|
||||
ConstrainRAMSpace=no
|
||||
@@ -1,193 +0,0 @@
|
||||
#
|
||||
# See the slurm.conf man page for more information.
|
||||
#
|
||||
ControlMachine=slurm-ctl
|
||||
# ControlAddr=127.0.0.1
|
||||
#BackupController=
|
||||
#BackupAddr=
|
||||
#
|
||||
AuthType=auth/munge
|
||||
#CheckpointType=checkpoint/none
|
||||
CryptoType=crypto/munge
|
||||
CommunicationParameters=NoAddrCache
|
||||
SlurmctldParameters=enable_configless
|
||||
#DisableRootJobs=NO
|
||||
#EnforcePartLimits=NO
|
||||
#Epilog=
|
||||
#EpilogSlurmctld=
|
||||
#FirstJobId=1
|
||||
#MaxJobId=999999
|
||||
#GresTypes=
|
||||
#GroupUpdateForce=0
|
||||
#GroupUpdateTime=600
|
||||
#JobCheckpointDir=/var/slurm/checkpoint
|
||||
#JobCredentialPrivateKey=
|
||||
#JobCredentialPublicCertificate=
|
||||
#JobFileAppend=0
|
||||
#JobRequeue=1
|
||||
#JobSubmitPlugins=
|
||||
#KillOnBadExit=0
|
||||
#LaunchType=launch/slurm
|
||||
#Licenses=foo*4,bar
|
||||
#MailProg=/bin/true
|
||||
#MaxJobCount=5000
|
||||
#MaxStepCount=40000
|
||||
#MaxTasksPerNode=128
|
||||
MpiDefault=pmix
|
||||
# when running slurmd as user change to: MpiDefault=none
|
||||
#MpiParams=ports=#-#
|
||||
#PluginDir=
|
||||
#PlugStackConfig=
|
||||
#PrivateData=jobs
|
||||
{% if container_privileged -%}
|
||||
ProctrackType=proctrack/cgroup
|
||||
{% else -%}
|
||||
{# when not running privileged docker container, cgroups cannot be used and
|
||||
jobs fail without comment in any log #}
|
||||
ProctrackType=proctrack/linuxproc
|
||||
{% endif %}
|
||||
#Prolog=
|
||||
#PrologFlags=
|
||||
#PrologSlurmctld=
|
||||
#PropagatePrioProcess=0
|
||||
#PropagateResourceLimits=
|
||||
#PropagateResourceLimitsExcept=
|
||||
#RebootProgram=
|
||||
ReturnToService=1
|
||||
#SallocDefaultCommand=
|
||||
SlurmctldPidFile=/var/run/slurm/slurmctld.pid
|
||||
SlurmctldPort=6817
|
||||
SlurmdPidFile=/var/run/slurm/slurmd.pid
|
||||
SlurmdPort=6818
|
||||
SlurmdSpoolDir=/var/spool/slurm/d
|
||||
SlurmUser={{slurm_user}}
|
||||
SlurmdUser=root
|
||||
# SlurmdUser=slurm -> sbatch does not work
|
||||
#SrunEpilog=
|
||||
#SrunProlog=
|
||||
StateSaveLocation=/var/spool/slurm/ctld
|
||||
SwitchType=switch/none
|
||||
#TaskEpilog=
|
||||
TaskPlugin=task/none
|
||||
#TaskPluginParam=
|
||||
#TaskProlog=
|
||||
#TopologyPlugin=topology/tree
|
||||
#TmpFS=/tmp
|
||||
#TrackWCKey=no
|
||||
#TreeWidth=
|
||||
#UnkillableStepProgram=
|
||||
#UsePAM=0
|
||||
#
|
||||
#
|
||||
# TIMERS
|
||||
#BatchStartTimeout=10
|
||||
#CompleteWait=0
|
||||
#EpilogMsgTime=2000
|
||||
#GetEnvTimeout=2
|
||||
#HealthCheckInterval=0
|
||||
#HealthCheckProgram=
|
||||
InactiveLimit=0
|
||||
KillWait=30
|
||||
#MessageTimeout=10
|
||||
#ResvOverRun=0
|
||||
MinJobAge=300
|
||||
#OverTimeLimit=0
|
||||
SlurmctldTimeout=120
|
||||
SlurmdTimeout=300
|
||||
#UnkillableStepTimeout=60
|
||||
#VSizeFactor=0
|
||||
Waittime=0
|
||||
#
|
||||
#
|
||||
# SCHEDULING
|
||||
#DefMemPerCPU=0
|
||||
#FastSchedule=1
|
||||
#MaxMemPerCPU=0
|
||||
#SchedulerTimeSlice=30
|
||||
SchedulerType=sched/backfill
|
||||
# SelectType=select/linear
|
||||
SelectType=select/cons_res
|
||||
#SelectTypeParameters=
|
||||
SelectTypeParameters=CR_CORE
|
||||
#
|
||||
#
|
||||
# JOB PRIORITY
|
||||
#PriorityFlags=
|
||||
#PriorityType=priority/basic
|
||||
#PriorityDecayHalfLife=
|
||||
#PriorityCalcPeriod=
|
||||
#PriorityFavorSmall=
|
||||
#PriorityMaxAge=
|
||||
#PriorityUsageResetPeriod=
|
||||
#PriorityWeightAge=
|
||||
#PriorityWeightFairshare=
|
||||
#PriorityWeightJobSize=
|
||||
#PriorityWeightPartition=
|
||||
#PriorityWeightQOS=
|
||||
#
|
||||
#
|
||||
# LOGGING AND ACCOUNTING
|
||||
#AccountingStorageEnforce=0
|
||||
#AccountingStorageHost=
|
||||
#AccountingStorageLoc=
|
||||
#AccountingStoragePass=
|
||||
#AccountingStoragePort=
|
||||
AccountingStorageType=accounting_storage/none
|
||||
#AccountingStorageUser=
|
||||
AccountingStoreJobComment=YES
|
||||
ClusterName=cluster
|
||||
#DebugFlags=Steps,TraceJobs
|
||||
#JobCompHost=
|
||||
JobCompLoc=/tmp/jobcomp
|
||||
#JobCompPass=
|
||||
#JobCompPort=
|
||||
JobCompType=jobcomp/filetxt
|
||||
#JobCompUser=
|
||||
#JobContainerType=job_container/none
|
||||
JobAcctGatherFrequency=30
|
||||
JobAcctGatherType=jobacct_gather/none
|
||||
SlurmctldDebug=verbose
|
||||
SlurmctldLogFile={{slurm_log_path_ctld}}
|
||||
SlurmdDebug=verbose
|
||||
SlurmdLogFile={{slurm_log_path_d}}
|
||||
SlurmSchedLogFile={{slurm_log_path_sched}}
|
||||
#SlurmSchedLogLevel=
|
||||
#
|
||||
#
|
||||
# POWER SAVE SUPPORT FOR IDLE NODES (optional)
|
||||
#SuspendProgram=
|
||||
#ResumeProgram=
|
||||
#SuspendTimeout=
|
||||
#ResumeTimeout=
|
||||
#ResumeRate=
|
||||
#SuspendExcNodes=
|
||||
#SuspendExcParts=
|
||||
#SuspendRate=
|
||||
#SuspendTime=
|
||||
#
|
||||
#
|
||||
# COMPUTE NODES
|
||||
NodeName=slurm-exec[1-{{ num_nodes }}] CPUs={{ slurm_exec_node_cores }} {{''
|
||||
}} RealMemory={{ slurm_exec_node_mem }} {{''
|
||||
}} CoresPerSocket={{ slurm_exec_node_cores }} State=UNKNOWN
|
||||
{% for p in partitions | default([]) %}
|
||||
NodeName={{ p.nodeprefix }}[1-{{ p.num_nodes }}] CPUs={{ p.node_cores }} {{''
|
||||
}} RealMemory={{ p.node_mem }} {{''
|
||||
}} CoresPerSocket={{ p.node_cores }} {{''
|
||||
}} {%- if p.port is defined %} Port={{ p.port}} {% endif %}{{''
|
||||
}} State={{ p.initstate | default('UNKNOWN') }}
|
||||
{% endfor %}
|
||||
{% for i in alloc_nodes -%}
|
||||
NodeName={{i.name}}
|
||||
{%- if i.hostname is defined %} NodeHostname={{i.hostname}} {% endif %}
|
||||
{%- if i.addr is defined %} NodeAddr={{i.addr}} {% endif %}
|
||||
State=UNKNOWN
|
||||
{% endfor %}
|
||||
PartitionName=container Nodes=slurm-exec[1-{{num_nodes}}] {{ ''
|
||||
}} AllocNodes={{alloc_nodes |map(attribute='name') | join(',')}} {{ ''
|
||||
}} Default=YES MaxTime=INFINITE State=UP
|
||||
{% for p in partitions | default([]) %}
|
||||
PartitionName={{ p.name }} Nodes={{ p.nodeprefix }}[1-{{ p.num_nodes }}] {{ ''
|
||||
}} AllocNodes={{alloc_nodes |map(attribute='name') | join(',')}} {{ ''
|
||||
}} MaxTime=INFINITE State=UP
|
||||
{% endfor %}
|
||||
@@ -1,21 +0,0 @@
|
||||
slurm_nodes_std: # default nodes: controller and submit machine
|
||||
- machine: ctl
|
||||
image: slurm:slurmctld
|
||||
exposed_ports: [ "6817:6817/tcp" ]
|
||||
- machine: submit1
|
||||
image: slurm:slurmd
|
||||
extra_mounts:
|
||||
- "/home/{{unpriv_user}}/job3/:/mnt/:rw"
|
||||
slurm_nodes_exec: | # extend range to execute nodes list
|
||||
{% set slurm_nodes_exec = [] %}
|
||||
{% for i in range(1, num_nodes+1) -%}
|
||||
{% set _ = slurm_nodes_exec.extend([
|
||||
{'machine':'exec%s'|format(i), 'image': 'slurm:slurmd',
|
||||
'aliases':['drone%s'|format(i)]}]) -%}
|
||||
{%- endfor %}
|
||||
{{ slurm_nodes_exec }}
|
||||
slurm_default_mounts:
|
||||
- /container/volumes/slurm/:/etc/slurm/:rw
|
||||
- /container/volumes/munge/munge.key:/etc/munge/munge.key:rw
|
||||
- slurm-shared:/shared/:rw
|
||||
|
||||
Reference in New Issue
Block a user