StarMirror
/
podman
mirror of https://github.com/containers/podman
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

kube play: exit-code propagation 

Implement means for reflecting failed containers (i.e., those having
exited non-zero) to better integrate `kube play` with systemd.  The
idea is to have the main PID of `kube play` exit non-zero in a
configurable way such that systemd's restart policies can kick in.

When using the default sdnotify-notify policy, the service container
acts as the main PID to further reduce the resource footprint.  In that
case, before stopping the service container, Podman will lookup the exit
codes of all non-infra containers.  The service will then behave
according to the following three exit-code policies:

 - `none`: exit 0 and ignore containers (default)
 - `any`: exit non-zero if _any_ container did
 - `all`: exit non-zero if _all_ containers did

The upper values can be passed via a hidden `kube play
--service-exit-code-propagation` flag which can be used by tests and
later on by Quadlet.

In case Podman acts as the main PID (i.e., when at least one container
runs with an sdnotify-policy other than "ignore"), Podman will continue
to wait for the service container to exit and reflect its exit code.

Note that this commit also fixes a long-standing annoyance of the
service container exiting non-zero.  The underlying issue was that the
service container had been stopped with SIGKILL instead of SIGTERM and
hence exited non-zero.  Fixing that was a prerequisite for the exit-code
propagation to work but also improves the integration of `kube play`
with systemd and hence Quadlet with systemd.

Jira: issues.redhat.com/browse/RUN-1776
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
pull/18671/head
Valentin Rothberg 2023-05-22 17:21:53 +02:00
parent 6dbc138339
commit 08b0d93ea3
14 changed files with 352 additions and 143 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
cmd/podman/kube/play.go
View File

@ -181,18 +181,19 @@ func playFlags(cmd *cobra.Command) {
flags.StringVar(&playOptions.ContextDir, contextDirFlagName, "", "Path to top level of context directory") flags.StringVar(&playOptions.ContextDir, contextDirFlagName, "", "Path to top level of context directory")
_ = cmd.RegisterFlagCompletionFunc(contextDirFlagName, completion.AutocompleteDefault) _ = cmd.RegisterFlagCompletionFunc(contextDirFlagName, completion.AutocompleteDefault)
// NOTE: The service-container flag is marked as hidden as it
// is purely designed for running kube-play or play-kube in systemd units.
// It is not something users should need to know or care about.
//
// Having a flag rather than an env variable is cleaner.
serviceFlagName := "service-container"
flags.BoolVar(&playOptions.ServiceContainer, serviceFlagName, false, "Starts a service container before all pods")
_ = flags.MarkHidden("service-container")
flags.StringVar(&playOptions.SignaturePolicy, "signature-policy", "", "`Pathname` of signature policy file (not usually used)") flags.StringVar(&playOptions.SignaturePolicy, "signature-policy", "", "`Pathname` of signature policy file (not usually used)")
_ = flags.MarkHidden("signature-policy") _ = flags.MarkHidden("signature-policy")
// Below flags are local-only and hidden since they are used in
// kube-play's systemd integration only and hence hidden from
// users.
serviceFlagName := "service-container"
flags.BoolVar(&playOptions.ServiceContainer, serviceFlagName, false, "Starts a service container before all pods")
_ = flags.MarkHidden(serviceFlagName)
exitFlagName := "service-exit-code-propagation"
flags.StringVar(&playOptions.ExitCodePropagation, exitFlagName, "", "Exit-code propagation of the service container")
_ = flags.MarkHidden(exitFlagName)
} }
} }
@ -450,6 +451,9 @@ func kubeplay(body io.Reader) error {
if err != nil { if err != nil {
return err return err
} }
if report.ExitCode != nil {
registry.SetExitCode(int(*report.ExitCode))
}
if err := printPlayReport(report); err != nil { if err := printPlayReport(report); err != nil {
return err return err
} }

81
docs/source/markdown/podman-container-inspect.1.md.in
View File

@ -20,46 +20,47 @@ The keys of the returned JSON can be used as the values for the --format flag (s
Valid placeholders for the Go template are listed below: Valid placeholders for the Go template are listed below:
| **Placeholder** | **Description** | | **Placeholder** | **Description** |
| ----------------- | ------------------ | | ------------------------ | -------------------------------------------------- |
| .AppArmorProfile | AppArmor profile (string) | | .AppArmorProfile | AppArmor profile (string) |
| .Args | Command-line arguments (array of strings) | | .Args | Command-line arguments (array of strings) |
| .BoundingCaps | Bounding capability set (array of strings) | | .BoundingCaps | Bounding capability set (array of strings) |
| .Config ... | Structure with config info | | .Config ... | Structure with config info |
| .ConmonPidFile | Path to file containing conmon pid (string) | | .ConmonPidFile | Path to file containing conmon pid (string) |
| .Created | Container creation time (string, ISO3601) | | .Created | Container creation time (string, ISO3601) |
| .Dependencies | Dependencies (array of strings) | | .Dependencies | Dependencies (array of strings) |
| .Driver | Storage driver (string) | | .Driver | Storage driver (string) |
| .EffectiveCaps | Effective capability set (array of strings) | | .EffectiveCaps | Effective capability set (array of strings) |
| .ExecIDs | Exec IDs (array of strings) | | .ExecIDs | Exec IDs (array of strings) |
| .GraphDriver ... | Further details of graph driver (struct) | | .GraphDriver ... | Further details of graph driver (struct) |
| .HostConfig ... | Host config details (struct) | | .HostConfig ... | Host config details (struct) |
| .HostnamePath | Path to file containing hostname (string) | | .HostnamePath | Path to file containing hostname (string) |
| .HostsPath | Path to container /etc/hosts file (string) | | .HostsPath | Path to container /etc/hosts file (string) |
| .ID | Container ID (full 64-char hash) | | .ID | Container ID (full 64-char hash) |
| .Image | Container image ID (64-char hash) | | .Image | Container image ID (64-char hash) |
| .ImageDigest | Container image digest (sha256:+64-char hash) | | .ImageDigest | Container image digest (sha256:+64-char hash) |
| .ImageName | Container image name (string) | | .ImageName | Container image name (string) |
| .IsInfra | Is this an infra container? (string: true/false) | | .IsInfra | Is this an infra container? (string: true/false) |
| .IsService | Is this a service container? (string: true/false) | | .IsService | Is this a service container? (string: true/false) |
| .MountLabel | SELinux label of mount (string) | | .KubeExitCodePropagation | Kube exit-code propagation (string) |
| .Mounts | Mounts (array of strings) | | .MountLabel | SELinux label of mount (string) |
| .Name | Container name (string) | | .Mounts | Mounts (array of strings) |
| .Namespace | Container namespace (string) | | .Name | Container name (string) |
| .NetworkSettings ... | Network settings (struct) | | .Namespace | Container namespace (string) |
| .OCIConfigPath | Path to OCI config file (string) | | .NetworkSettings ... | Network settings (struct) |
| .OCIRuntime | OCI runtime name (string) | | .OCIConfigPath | Path to OCI config file (string) |
| .Path | Path to container command (string) | | .OCIRuntime | OCI runtime name (string) |
| .PidFile | Path to file containing container PID (string) | | .Path | Path to container command (string) |
| .Pod | Parent pod (string) | | .PidFile | Path to file containing container PID (string) |
| .ProcessLabel | SELinux label of process (string) | | .Pod | Parent pod (string) |
| .ResolvConfPath | Path to container's resolv.conf file (string) | | .ProcessLabel | SELinux label of process (string) |
| .RestartCount | Number of times container has been restarted (int) | | .ResolvConfPath | Path to container's resolv.conf file (string) |
| .Rootfs | Container rootfs (string) | | .RestartCount | Number of times container has been restarted (int) |
| .SizeRootFs | Size of rootfs, in bytes [1] | | .Rootfs | Container rootfs (string) |
| .SizeRw | Size of upper (R/W) container layer, in bytes [1] | | .SizeRootFs | Size of rootfs, in bytes [1] |
| .State ... | Container state info (struct) | | .SizeRw | Size of upper (R/W) container layer, in bytes [1] |
| .StaticDir | Path to container metadata dir (string) | | .State ... | Container state info (struct) |
| .StaticDir | Path to container metadata dir (string) |
[1] This format specifier requires the **--size** option [1] This format specifier requires the **--size** option

2
libpod/container_config.go
View File

@ -360,6 +360,8 @@ type ContainerMiscConfig struct {
CgroupParent string `json:"cgroupParent"` CgroupParent string `json:"cgroupParent"`
// GroupEntry specifies arbitrary data to append to a file. // GroupEntry specifies arbitrary data to append to a file.
GroupEntry string `json:"group_entry,omitempty"` GroupEntry string `json:"group_entry,omitempty"`
// KubeExitCodePropagation of the service container.
KubeExitCodePropagation define.KubeExitCodePropagation `json:"kubeExitCodePropagation"`
// LogPath log location // LogPath log location
LogPath string `json:"logPath"` LogPath string `json:"logPath"`
// LogTag is the tag used for logging // LogTag is the tag used for logging

49
libpod/container_inspect.go
View File

@ -141,30 +141,31 @@ func (c *Container) getContainerInspectData(size bool, driverData *define.Driver
CheckpointLog: runtimeInfo.CheckpointLog, CheckpointLog: runtimeInfo.CheckpointLog,
RestoreLog: runtimeInfo.RestoreLog, RestoreLog: runtimeInfo.RestoreLog,
}, },
Image: config.RootfsImageID, Image: config.RootfsImageID,
ImageName: config.RootfsImageName, ImageName: config.RootfsImageName,
Namespace: config.Namespace, Namespace: config.Namespace,
Rootfs: config.Rootfs, Rootfs: config.Rootfs,
Pod: config.Pod, Pod: config.Pod,
ResolvConfPath: resolvPath, ResolvConfPath: resolvPath,
HostnamePath: hostnamePath, HostnamePath: hostnamePath,
HostsPath: hostsPath, HostsPath: hostsPath,
StaticDir: config.StaticDir, StaticDir: config.StaticDir,
OCIRuntime: config.OCIRuntime, OCIRuntime: config.OCIRuntime,
ConmonPidFile: config.ConmonPidFile, ConmonPidFile: config.ConmonPidFile,
PidFile: config.PidFile, PidFile: config.PidFile,
Name: config.Name, Name: config.Name,
RestartCount: int32(runtimeInfo.RestartCount), RestartCount: int32(runtimeInfo.RestartCount),
Driver: driverData.Name, Driver: driverData.Name,
MountLabel: config.MountLabel, MountLabel: config.MountLabel,
ProcessLabel: config.ProcessLabel, ProcessLabel: config.ProcessLabel,
AppArmorProfile: ctrSpec.Process.ApparmorProfile, AppArmorProfile: ctrSpec.Process.ApparmorProfile,
ExecIDs: execIDs, ExecIDs: execIDs,
GraphDriver: driverData, GraphDriver: driverData,
Mounts: inspectMounts, Mounts: inspectMounts,
Dependencies: c.Dependencies(), Dependencies: c.Dependencies(),
IsInfra: c.IsInfra(), IsInfra: c.IsInfra(),
IsService: c.IsService(), IsService: c.IsService(),
KubeExitCodePropagation: config.KubeExitCodePropagation.String(),
} }
if config.RootfsImageID != "" { // May not be set if the container was created with --rootfs if config.RootfsImageID != "" { // May not be set if the container was created with --rootfs

77
libpod/define/container_inspect.go
View File

@ -654,44 +654,45 @@ type InspectNetworkSettings struct {
// compatible with `docker inspect` JSON, but additional fields have been added // compatible with `docker inspect` JSON, but additional fields have been added
// as required to share information not in the original output. // as required to share information not in the original output.
type InspectContainerData struct { type InspectContainerData struct {
ID string `json:"Id"` ID string `json:"Id"`
Created time.Time `json:"Created"` Created time.Time `json:"Created"`
Path string `json:"Path"` Path string `json:"Path"`
Args []string `json:"Args"` Args []string `json:"Args"`
State *InspectContainerState `json:"State"` State *InspectContainerState `json:"State"`
Image string `json:"Image"` Image string `json:"Image"`
ImageDigest string `json:"ImageDigest"` ImageDigest string `json:"ImageDigest"`
ImageName string `json:"ImageName"` ImageName string `json:"ImageName"`
Rootfs string `json:"Rootfs"` Rootfs string `json:"Rootfs"`
Pod string `json:"Pod"` Pod string `json:"Pod"`
ResolvConfPath string `json:"ResolvConfPath"` ResolvConfPath string `json:"ResolvConfPath"`
HostnamePath string `json:"HostnamePath"` HostnamePath string `json:"HostnamePath"`
HostsPath string `json:"HostsPath"` HostsPath string `json:"HostsPath"`
StaticDir string `json:"StaticDir"` StaticDir string `json:"StaticDir"`
OCIConfigPath string `json:"OCIConfigPath,omitempty"` OCIConfigPath string `json:"OCIConfigPath,omitempty"`
OCIRuntime string `json:"OCIRuntime,omitempty"` OCIRuntime string `json:"OCIRuntime,omitempty"`
ConmonPidFile string `json:"ConmonPidFile"` ConmonPidFile string `json:"ConmonPidFile"`
PidFile string `json:"PidFile"` PidFile string `json:"PidFile"`
Name string `json:"Name"` Name string `json:"Name"`
RestartCount int32 `json:"RestartCount"` RestartCount int32 `json:"RestartCount"`
Driver string `json:"Driver"` Driver string `json:"Driver"`
MountLabel string `json:"MountLabel"` MountLabel string `json:"MountLabel"`
ProcessLabel string `json:"ProcessLabel"` ProcessLabel string `json:"ProcessLabel"`
AppArmorProfile string `json:"AppArmorProfile"` AppArmorProfile string `json:"AppArmorProfile"`
EffectiveCaps []string `json:"EffectiveCaps"` EffectiveCaps []string `json:"EffectiveCaps"`
BoundingCaps []string `json:"BoundingCaps"` BoundingCaps []string `json:"BoundingCaps"`
ExecIDs []string `json:"ExecIDs"` ExecIDs []string `json:"ExecIDs"`
GraphDriver *DriverData `json:"GraphDriver"` GraphDriver *DriverData `json:"GraphDriver"`
SizeRw *int64 `json:"SizeRw,omitempty"` SizeRw *int64 `json:"SizeRw,omitempty"`
SizeRootFs int64 `json:"SizeRootFs,omitempty"` SizeRootFs int64 `json:"SizeRootFs,omitempty"`
Mounts []InspectMount `json:"Mounts"` Mounts []InspectMount `json:"Mounts"`
Dependencies []string `json:"Dependencies"` Dependencies []string `json:"Dependencies"`
NetworkSettings *InspectNetworkSettings `json:"NetworkSettings"` NetworkSettings *InspectNetworkSettings `json:"NetworkSettings"`
Namespace string `json:"Namespace"` Namespace string `json:"Namespace"`
IsInfra bool `json:"IsInfra"` IsInfra bool `json:"IsInfra"`
IsService bool `json:"IsService"` IsService bool `json:"IsService"`
Config *InspectContainerConfig `json:"Config"` KubeExitCodePropagation string `json:"KubeExitCodePropagation"`
HostConfig *InspectContainerHostConfig `json:"HostConfig"` Config *InspectContainerConfig `json:"Config"`
HostConfig *InspectContainerHostConfig `json:"HostConfig"`
} }
// InspectExecSession contains information about a given exec session. // InspectExecSession contains information about a given exec session.

54
libpod/define/exit_code_propagation.go Normal file
View File

@ -0,0 +1,54 @@
package define
import "fmt"
// KubeExitCodePropagation defines an exit policy of kube workloads.
type KubeExitCodePropagation int
const (
// Invalid exit policy for a proper type system.
KubeExitCodePropagationInvalid KubeExitCodePropagation = iota
// Exit 0 regardless of any failed containers.
KubeExitCodePropagationNone
// Exit non-zero if all containers failed.
KubeExitCodePropagationAll
// Exit non-zero if any container failed.
KubeExitCodePropagationAny
// String representations.
strKubeECPInvalid = "invalid"
strKubeECPNone = "none"
strKubeECPAll = "all"
strKubeECPAny = "any"
)
// Parse the specified kube exit-code propagation. Return an error if an
// unsupported value is specified.
func ParseKubeExitCodePropagation(value string) (KubeExitCodePropagation, error) {
switch value {
case strKubeECPNone, "":
return KubeExitCodePropagationNone, nil
case strKubeECPAll:
return KubeExitCodePropagationAll, nil
case strKubeECPAny:
return KubeExitCodePropagationAny, nil
default:
return KubeExitCodePropagationInvalid, fmt.Errorf("unsupported exit-code propagation %q", value)
}
}
// Return the string representation of the KubeExitCodePropagation.
func (k KubeExitCodePropagation) String() string {
switch k {
case KubeExitCodePropagationNone:
return strKubeECPNone
case KubeExitCodePropagationAll:
return strKubeECPAll
case KubeExitCodePropagationAny:
return strKubeECPAny
case KubeExitCodePropagationInvalid:
return strKubeECPInvalid
default:
return "unknown value"
}
}

7
libpod/options.go
View File

@ -1579,15 +1579,16 @@ func withIsInfra() CtrCreateOption {
} }
// WithIsService allows us to differentiate between service containers and other container // WithIsService allows us to differentiate between service containers and other container
// within the container config // within the container config. It also sets the exit-code propagation of the
func WithIsService() CtrCreateOption { // service container.
func WithIsService(ecp define.KubeExitCodePropagation) CtrCreateOption {
return func(ctr *Container) error { return func(ctr *Container) error {
if ctr.valid { if ctr.valid {
return define.ErrCtrFinalized return define.ErrCtrFinalized
} }
ctr.config.IsService = true ctr.config.IsService = true
ctr.config.KubeExitCodePropagation = ecp
return nil return nil
} }
} }

4
libpod/runtime_ctr.go
View File

@ -957,11 +957,11 @@ func (r *Runtime) evictContainer(ctx context.Context, idOrName string, removeVol
} }
if c.IsService() { if c.IsService() {
canStop, err := c.canStopServiceContainer() report, err := c.canStopServiceContainer()
if err != nil { if err != nil {
return id, err return id, err
} }
if !canStop { if !report.canBeStopped {
return id, fmt.Errorf("container %s is the service container of pod(s) %s and cannot be removed without removing the pod(s)", c.ID(), strings.Join(c.state.Service.Pods, ",")) return id, fmt.Errorf("container %s is the service container of pod(s) %s and cannot be removed without removing the pod(s)", c.ID(), strings.Join(c.state.Service.Pods, ","))
} }
} }

101
libpod/service.go
View File

@ -7,6 +7,7 @@ import (
"github.com/containers/podman/v4/libpod/define" "github.com/containers/podman/v4/libpod/define"
"github.com/sirupsen/logrus" "github.com/sirupsen/logrus"
"golang.org/x/sys/unix"
) )
// A service consists of one or more pods. The service container is started // A service consists of one or more pods. The service container is started
@ -59,17 +60,29 @@ func (c *Container) IsService() bool {
return c.config.IsService return c.config.IsService
} }
// serviceContainerReport bundles information when checking whether a service
// container can be stopped.
type serviceContainerReport struct {
// Indicates whether the service container can be stopped or not.
canBeStopped bool
// Number of all known containers below the service container.
numContainers int
// Number of containers below the service containers that exited
// non-zero.
failedContainers int
}
// canStopServiceContainerLocked returns true if all pods of the service are stopped. // canStopServiceContainerLocked returns true if all pods of the service are stopped.
// Note that the method acquires the container lock. // Note that the method acquires the container lock.
func (c *Container) canStopServiceContainerLocked() (bool, error) { func (c *Container) canStopServiceContainerLocked() (*serviceContainerReport, error) {
c.lock.Lock() c.lock.Lock()
defer c.lock.Unlock() defer c.lock.Unlock()
if err := c.syncContainer(); err != nil { if err := c.syncContainer(); err != nil {
return false, err return nil, err
} }
if !c.IsService() { if !c.IsService() {
return false, fmt.Errorf("internal error: checking service: container %s is not a service container", c.ID()) return nil, fmt.Errorf("internal error: checking service: container %s is not a service container", c.ID())
} }
return c.canStopServiceContainer() return c.canStopServiceContainer()
@ -77,14 +90,15 @@ func (c *Container) canStopServiceContainerLocked() (bool, error) {
// canStopServiceContainer returns true if all pods of the service are stopped. // canStopServiceContainer returns true if all pods of the service are stopped.
// Note that the method expects the container to be locked. // Note that the method expects the container to be locked.
func (c *Container) canStopServiceContainer() (bool, error) { func (c *Container) canStopServiceContainer() (*serviceContainerReport, error) {
report := serviceContainerReport{canBeStopped: true}
for _, id := range c.state.Service.Pods { for _, id := range c.state.Service.Pods {
pod, err := c.runtime.LookupPod(id) pod, err := c.runtime.LookupPod(id)
if err != nil { if err != nil {
if errors.Is(err, define.ErrNoSuchPod) { if errors.Is(err, define.ErrNoSuchPod) {
continue continue
} }
return false, err return nil, err
} }
status, err := pod.GetPodStatus() status, err := pod.GetPodStatus()
@ -92,19 +106,37 @@ func (c *Container) canStopServiceContainer() (bool, error) {
if errors.Is(err, define.ErrNoSuchPod) { if errors.Is(err, define.ErrNoSuchPod) {
continue continue
} }
return false, err return nil, err
} }
// We can only stop the service if all pods are done.
switch status { switch status {
case define.PodStateStopped, define.PodStateExited, define.PodStateErrored: case define.PodStateStopped, define.PodStateExited, define.PodStateErrored:
continue podCtrs, err := c.runtime.state.PodContainers(pod)
if err != nil {
return nil, err
}
for _, pc := range podCtrs {
if pc.IsInfra() {
continue // ignore infra containers
}
exitCode, err := c.runtime.state.GetContainerExitCode(pc.ID())
if err != nil {
return nil, err
}
if exitCode != 0 {
report.failedContainers++
}
report.numContainers++
}
default: default:
return false, nil // Service container cannot be stopped, so we can
// return early.
report.canBeStopped = false
return &report, nil
} }
} }
return true, nil return &report, nil
} }
// Checks whether the service container can be stopped and does so. // Checks whether the service container can be stopped and does so.
@ -125,21 +157,49 @@ func (p *Pod) maybeStopServiceContainer() error {
// pod->container->servicePods hierarchy. // pod->container->servicePods hierarchy.
p.runtime.queueWork(func() { p.runtime.queueWork(func() {
logrus.Debugf("Pod %s has a service %s: checking if it can be stopped", p.ID(), serviceCtr.ID()) logrus.Debugf("Pod %s has a service %s: checking if it can be stopped", p.ID(), serviceCtr.ID())
canStop, err := serviceCtr.canStopServiceContainerLocked() report, err := serviceCtr.canStopServiceContainerLocked()
if err != nil { if err != nil {
logrus.Errorf("Checking whether service of container %s can be stopped: %v", serviceCtr.ID(), err) logrus.Errorf("Checking whether service of container %s can be stopped: %v", serviceCtr.ID(), err)
return return
} }
if !canStop { if !report.canBeStopped {
return return
} }
logrus.Debugf("Stopping service container %s", serviceCtr.ID())
if err := serviceCtr.Stop(); err != nil && !errors.Is(err, define.ErrCtrStopped) { // Now either kill or stop the service container, depending on the configured exit policy.
// Log this in debug mode so that we don't print out an error and confuse the user stop := func() {
// when the service container can't be stopped because it is in created state // Note that the service container runs catatonit which
// This can happen when an error happens during kube play and we are trying to // will exit gracefully on SIGINT.
// clean up after the error. logrus.Debugf("Stopping service container %s", serviceCtr.ID())
logrus.Debugf("Error stopping service container %s: %v", serviceCtr.ID(), err) if err := serviceCtr.Kill(uint(unix.SIGINT)); err != nil && !errors.Is(err, define.ErrCtrStateInvalid) {
logrus.Debugf("Error stopping service container %s: %v", serviceCtr.ID(), err)
}
}
kill := func() {
logrus.Debugf("Killing service container %s", serviceCtr.ID())
if err := serviceCtr.Kill(uint(unix.SIGKILL)); err != nil && !errors.Is(err, define.ErrCtrStateInvalid) {
logrus.Debugf("Error killing service container %s: %v", serviceCtr.ID(), err)
}
}
switch serviceCtr.config.KubeExitCodePropagation {
case define.KubeExitCodePropagationNone:
stop()
case define.KubeExitCodePropagationAny:
if report.failedContainers > 0 {
kill()
} else {
stop()
}
case define.KubeExitCodePropagationAll:
if report.failedContainers == report.numContainers {
kill()
} else {
stop()
}
default:
logrus.Errorf("Internal error: cannot stop service container %s: unknown exit policy %q", serviceCtr.ID(), serviceCtr.config.KubeExitCodePropagation.String())
} }
}) })
return nil return nil
@ -240,9 +300,8 @@ func (p *Pod) maybeRemoveServiceContainer() error {
if !canRemove { if !canRemove {
return return
} }
timeout := uint(0)
logrus.Debugf("Removing service container %s", serviceCtr.ID()) logrus.Debugf("Removing service container %s", serviceCtr.ID())
if err := p.runtime.RemoveContainer(context.Background(), serviceCtr, true, false, &timeout); err != nil { if err := p.runtime.RemoveContainer(context.Background(), serviceCtr, true, false, nil); err != nil {
if !errors.Is(err, define.ErrNoSuchCtr) { if !errors.Is(err, define.ErrNoSuchCtr) {
logrus.Errorf("Removing service container %s: %v", serviceCtr.ID(), err) logrus.Errorf("Removing service container %s: %v", serviceCtr.ID(), err)
} }

5
pkg/domain/entities/play.go
View File

@ -21,6 +21,9 @@ type PlayKubeOptions struct {
// Down indicates whether to bring contents of a yaml file "down" // Down indicates whether to bring contents of a yaml file "down"
// as in stop // as in stop
Down bool Down bool
// ExitCodePropagation decides how the main PID of the Kube service
// should exit depending on the containers' exit codes.
ExitCodePropagation string
// Replace indicates whether to delete and recreate a yaml file // Replace indicates whether to delete and recreate a yaml file
Replace bool Replace bool
// Do not create /etc/hosts within the pod's containers, // Do not create /etc/hosts within the pod's containers,
@ -100,6 +103,8 @@ type PlayKubeReport struct {
Secrets []PlaySecret Secrets []PlaySecret
// ServiceContainerID - ID of the service container if one is created // ServiceContainerID - ID of the service container if one is created
ServiceContainerID string ServiceContainerID string
// If set, exit with the specified exit code.
ExitCode *int32
} }
type KubePlayReport = PlayKubeReport type KubePlayReport = PlayKubeReport

11
pkg/domain/infra/abi/play.go
View File

@ -86,7 +86,12 @@ func (ic *ContainerEngine) createServiceContainer(ctx context.Context, name stri
if err != nil { if err != nil {
return nil, fmt.Errorf("creating runtime spec for service container: %w", err) return nil, fmt.Errorf("creating runtime spec for service container: %w", err)
} }
opts = append(opts, libpod.WithIsService())
ecp, err := define.ParseKubeExitCodePropagation(options.ExitCodePropagation)
if err != nil {
return nil, err
}
opts = append(opts, libpod.WithIsService(ecp))
// Set the sd-notify mode to "ignore". Podman is responsible for // Set the sd-notify mode to "ignore". Podman is responsible for
// sending the notify messages when all containers are ready. // sending the notify messages when all containers are ready.
@ -348,9 +353,11 @@ func (ic *ContainerEngine) PlayKube(ctx context.Context, body io.Reader, options
if err := notifyproxy.SendMessage("", message); err != nil { if err := notifyproxy.SendMessage("", message); err != nil {
return nil, err return nil, err
} }
if _, err := serviceContainer.Wait(ctx); err != nil { exitCode, err := serviceContainer.Wait(ctx)
if err != nil {
return nil, fmt.Errorf("waiting for service container: %w", err) return nil, fmt.Errorf("waiting for service container: %w", err)
} }
report.ExitCode = &exitCode
} }
report.ServiceContainerID = serviceContainer.ID() report.ServiceContainerID = serviceContainer.ID()

3
test/system/250-systemd.bats
View File

@ -459,8 +459,7 @@ $name stderr" "logs work with passthrough"
fi fi
sleep 0.5 sleep 0.5
done done
# The service is marked as failed as the service container exits non-zero. is "$output" "inactive" "systemd service transitioned to 'inactive' state: $service_name"
is "$output" "failed" "systemd service transitioned to 'inactive' state: $service_name"
# Now stop and start the service again. # Now stop and start the service again.
systemctl stop $service_name systemctl stop $service_name

3
test/system/252-quadlet.bats
View File

@ -439,8 +439,7 @@ EOF
run_podman container inspect --format "{{.State.Status}}" test_pod-test run_podman container inspect --format "{{.State.Status}}" test_pod-test
is "$output" "running" "container should be started by systemd and hence be running" is "$output" "running" "container should be started by systemd and hence be running"
# The service is marked as failed as the service container exits non-zero. service_cleanup $QUADLET_SERVICE_NAME inactive
service_cleanup $QUADLET_SERVICE_NAME failed
run_podman rmi $(pause_image) run_podman rmi $(pause_image)
} }

76
test/system/260-sdnotify.bats
View File

@ -371,4 +371,80 @@ READY=1" "sdnotify sent MAINPID and READY"
run_podman rmi $(pause_image) run_podman rmi $(pause_image)
} }
function generate_exit_code_yaml {
local fname=$1
local cmd1=$2
local cmd2=$3
local sdnotify_policy=$4
echo "
apiVersion: v1
kind: Pod
metadata:
labels:
app: test
name: test_pod
annotations:
io.containers.sdnotify: "$sdnotify_policy"
spec:
restartPolicy: Never
containers:
- name: ctr1
image: $IMAGE
command:
- $cmd1
- name: ctr2
image: $IMAGE
command:
- $cmd2
" > $fname
}
@test "podman kube play - exit-code propagation" {
fname=$PODMAN_TMPDIR/$(random_string).yaml
# Create a test matrix with the following arguments:
# exit-code propagation | ctr1 command | ctr2 command | service-container exit code
exit_tests="
all | true | true | 0
all | true | false | 0
all | false | false | 137
any | true | true | 0
any | false | true | 137
any | false | false | 137
none | true | true | 0
none | true | false | 0
none | false | false | 0
"
# I am sorry, this is a long test as we need to test the upper matrix
# twice. The first run is using the default sdnotify policy of "ignore".
# In this case, the service container serves as the main PID of the service
# to have a minimal resource footprint. The second run is using the
# "conmon" sdnotify policy in which case Podman needs to serve as the main
# PID to act as an sdnotify proxy; there Podman will wait for the service
# container to exit and reflects its exit code.
while read exit_code_prop cmd1 cmd2 exit_code; do
for sdnotify_policy in ignore conmon; do
generate_exit_code_yaml $fname $cmd1 $cmd2 $sdnotify_policy
yaml_sha=$(sha256sum $fname)
service_container="${yaml_sha:0:12}-service"
podman_exit=$exit_code
if [[ $sdnotify_policy == "ignore" ]];then
podman_exit=0
fi
run_podman $podman_exit kube play --service-exit-code-propagation="$exit_code_prop" --service-container $fname
run_podman container inspect --format '{{.KubeExitCodePropagation}}' $service_container
is "$output" "$exit_code_prop" "service container has the expected policy set in its annotations"
run_podman wait $service_container
is "$output" "$exit_code" "service container reflects expected exit code $exit_code (policy: $policy, cmd1: $cmd1, cmd2: $cmd2)"
run_podman kube down $fname
done
done < <(parse_table "$exit_tests")
# A final smoke test to make sure bogus policies lead to an error
run_podman 125 kube play --service-exit-code-propagation=bogus --service-container $fname
is "$output" "Error: unsupported exit-code propagation \"bogus\"" "error on unsupported exit-code propagation"
run_podman rmi $(pause_image)
}
# vim: filetype=sh # vim: filetype=sh