May 29 17:54:29.727606 ip-10-0-133-167 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 29 17:54:29.727621 ip-10-0-133-167 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 29 17:54:29.727630 ip-10-0-133-167 systemd[1]: kubelet.service: Failed with result 'resources'.
May 29 17:54:29.727944 ip-10-0-133-167 systemd[1]: Failed to start Kubernetes Kubelet.
May 29 17:54:39.833851 ip-10-0-133-167 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 29 17:54:39.833867 ip-10-0-133-167 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f963f3b05297435daaeceac63a6f82bd --
May 29 17:57:04.832461 ip-10-0-133-167 systemd[1]: Starting Kubernetes Kubelet...
May 29 17:57:05.256819 ip-10-0-133-167 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 17:57:05.256819 ip-10-0-133-167 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 29 17:57:05.256819 ip-10-0-133-167 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 17:57:05.256819 ip-10-0-133-167 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 29 17:57:05.256819 ip-10-0-133-167 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 17:57:05.258307 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.258211    2575 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 29 17:57:05.264963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264934    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 17:57:05.264963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264954    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 17:57:05.264963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264958    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 17:57:05.264963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264962    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 17:57:05.264963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264966    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 17:57:05.264963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264970    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 17:57:05.264963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264973    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264976    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264979    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264982    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264984    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264987    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264990    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264992    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264995    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.264997    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265000    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265002    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265005    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265007    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265010    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265013    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265015    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265018    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265027    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265030    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 17:57:05.265236 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265032    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265035    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265039    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265042    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265045    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265047    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265050    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265052    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265055    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265058    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265062    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265065    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265068    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265071    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265075    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265079    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265081    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265084    2575 feature_gate.go:328] unrecognized feature gate: Example
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265087    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 17:57:05.265772 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265089    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265092    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265094    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265097    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265099    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265102    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265105    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265107    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265110    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265113    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265115    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265119    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265122    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265124    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265126    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265129    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265131    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265134    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265136    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265139    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 17:57:05.266255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265141    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265144    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265146    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265149    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265167    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265170    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265174    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265178    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265180    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265183    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265186    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265189    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265191    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265195    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265198    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265201    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265203    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265206    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265208    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265211    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 17:57:05.266738 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265213    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265604    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265609    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265612    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265614    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265617    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265620    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265622    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265625    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265627    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265630    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265633    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265636    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265638    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265642    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265646    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265649    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265651    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265654    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 17:57:05.267239 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265658    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265660    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265663    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265665    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265668    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265671    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265676    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265680    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265683    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265686    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265688    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265691    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265694    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265696    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265699    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265701    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265704    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265706    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265709    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265711    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265713    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 17:57:05.267692 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265716    2575 feature_gate.go:328] unrecognized feature gate: Example
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265718    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265721    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265723    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265726    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265728    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265731    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265733    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265736    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265738    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265741    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265744    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265747    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265750    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265752    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265756    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265758    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265760    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265764    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 17:57:05.268268 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265766    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265769    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265771    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265774    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265777    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265780    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265782    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265785    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265787    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265790    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265793    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265796    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265798    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265802    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265804    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265806    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265809    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265811    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265815    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 17:57:05.268748 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265819    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265822    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265825    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265828    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265830    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265833    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265836    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265838    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.265842    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266475    2575 flags.go:64] FLAG: --address="0.0.0.0"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266486    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266494    2575 flags.go:64] FLAG: --anonymous-auth="true"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266499    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266504    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266508    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266512    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266517    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266520    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266523    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266527    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266530    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266533    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 29 17:57:05.269224 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266537    2575 flags.go:64] FLAG: --cgroup-root=""
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266540    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266543    2575 flags.go:64] FLAG: --client-ca-file=""
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266546    2575 flags.go:64] FLAG: --cloud-config=""
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266548    2575 flags.go:64] FLAG: --cloud-provider="external"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266551    2575 flags.go:64] FLAG: --cluster-dns="[]"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266556    2575 flags.go:64] FLAG: --cluster-domain=""
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266558    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266561    2575 flags.go:64] FLAG: --config-dir=""
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266564    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266568    2575 flags.go:64] FLAG: --container-log-max-files="5"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266572    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266575    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266577    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266581    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266584    2575 flags.go:64] FLAG: --contention-profiling="false"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266587    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266590    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266593    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266596    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266600    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266603    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266606    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266609    2575 flags.go:64] FLAG: --enable-load-reader="false"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266613    2575 flags.go:64] FLAG: --enable-server="true"
May 29 17:57:05.269765 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266616    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266621    2575 flags.go:64] FLAG: --event-burst="100"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266624    2575 flags.go:64] FLAG: --event-qps="50"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266627    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266630    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266633    2575 flags.go:64] FLAG: --eviction-hard=""
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266637    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266640    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266643    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266646    2575 flags.go:64] FLAG: --eviction-soft=""
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266649    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266651    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266654    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266657    2575 flags.go:64] FLAG: --experimental-mounter-path=""
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266660    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266663    2575 flags.go:64] FLAG: --fail-swap-on="true"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266666    2575 flags.go:64] FLAG: --feature-gates=""
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266670    2575 flags.go:64] FLAG: --file-check-frequency="20s"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266673    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266675    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266678    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266681    2575 flags.go:64] FLAG: --healthz-port="10248"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266684    2575 flags.go:64] FLAG: --help="false"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266688    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-133-167.ec2.internal"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266691    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
May 29 17:57:05.270386 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266694    2575 flags.go:64] FLAG: --http-check-frequency="20s"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266697    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266700    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266703    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266707    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266710    2575 flags.go:64] FLAG: --image-service-endpoint=""
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266713    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266717    2575 flags.go:64] FLAG: --kube-api-burst="100"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266720    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266723    2575 flags.go:64] FLAG: --kube-api-qps="50"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266726    2575 flags.go:64] FLAG: --kube-reserved=""
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266729    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266732    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266735    2575 flags.go:64] FLAG: --kubelet-cgroups=""
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266738    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266740    2575 flags.go:64] FLAG: --lock-file=""
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266743    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266746    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266749    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266755    2575 flags.go:64] FLAG: --log-json-split-stream="false"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266757    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266760    2575 flags.go:64] FLAG: --log-text-split-stream="false"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266763    2575 flags.go:64] FLAG: --logging-format="text"
May 29 17:57:05.270983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266766    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266769    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266772    2575 flags.go:64] FLAG: --manifest-url=""
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266775    2575 flags.go:64] FLAG: --manifest-url-header=""
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266779    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266782    2575 flags.go:64] FLAG: --max-open-files="1000000"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266787    2575 flags.go:64] FLAG: --max-pods="110"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266790    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266793    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266796    2575 flags.go:64] FLAG: --memory-manager-policy="None"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266800    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266803    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266806    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266809    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266817    2575 flags.go:64] FLAG: --node-status-max-images="50"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266820    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266823    2575 flags.go:64] FLAG: --oom-score-adj="-999"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266827    2575 flags.go:64] FLAG: --pod-cidr=""
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266830    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266835    2575 flags.go:64] FLAG: --pod-manifest-path=""
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266838    2575 flags.go:64] FLAG: --pod-max-pids="-1"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266841    2575 flags.go:64] FLAG: --pods-per-core="0"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266844    2575 flags.go:64] FLAG: --port="10250"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266847    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 29 17:57:05.271560 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266850    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0891253d5d6c989ca"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266854    2575 flags.go:64] FLAG: --qos-reserved=""
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266857    2575 flags.go:64] FLAG: --read-only-port="10255"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266860    2575 flags.go:64] FLAG: --register-node="true"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266863    2575 flags.go:64] FLAG: --register-schedulable="true"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266866    2575 flags.go:64] FLAG: --register-with-taints=""
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266870    2575 flags.go:64] FLAG: --registry-burst="10"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266873    2575 flags.go:64] FLAG: --registry-qps="5"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266875    2575 flags.go:64] FLAG: --reserved-cpus=""
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266878    2575 flags.go:64] FLAG: --reserved-memory=""
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266882    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266885    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266888    2575 flags.go:64] FLAG: --rotate-certificates="false"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266891    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266894    2575 flags.go:64] FLAG: --runonce="false"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266897    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266900    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266904    2575 flags.go:64] FLAG: --seccomp-default="false"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266907    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266910    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266913    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266919    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266922    2575 flags.go:64] FLAG: --storage-driver-password="root"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266925    2575 flags.go:64] FLAG: --storage-driver-secure="false"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266928    2575 flags.go:64] FLAG: --storage-driver-table="stats"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266930    2575 flags.go:64] FLAG: --storage-driver-user="root"
May 29 17:57:05.272136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266934    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266937    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266940    2575 flags.go:64] FLAG: --system-cgroups=""
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266943    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266949    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266951    2575 flags.go:64] FLAG: --tls-cert-file=""
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266955    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266959    2575 flags.go:64] FLAG: --tls-min-version=""
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266961    2575 flags.go:64] FLAG: --tls-private-key-file=""
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266964    2575 flags.go:64] FLAG: --topology-manager-policy="none"
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266967    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266970    2575 flags.go:64] FLAG: --topology-manager-scope="container"
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266975    2575 flags.go:64] FLAG: --v="2"
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266979    2575 flags.go:64] FLAG: --version="false"
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266983    2575 flags.go:64] FLAG: --vmodule=""
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266987    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.266990    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267081    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267085    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267088    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267091    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267094    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267097    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 17:57:05.272833 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267100    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267103    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267105    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267108    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267112    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267115    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267117    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267120    2575 feature_gate.go:328] unrecognized feature gate: Example
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267122    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267125    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267128    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267131    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267133    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267136    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267139    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267141    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267144    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267146    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267149    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267163    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 17:57:05.273448 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267166    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267171    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267175    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267178    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267180    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267183    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267186    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267188    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267191    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267193    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267196    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267198    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267201    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267204    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267206    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267209    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267215    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267218    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267220    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 17:57:05.273963 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267223    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267226    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267228    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267231    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267234    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267237    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267239    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267242    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267245    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267247    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267251    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267255    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267258    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267260    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267263    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267266    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267269    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267271    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267274    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 17:57:05.274452 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267276    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267279    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267281    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267284    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267286    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267289    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267291    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267294    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267296    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267299    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267303    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267305    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267307    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267310    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267313    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267315    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267318    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267320    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267323    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 17:57:05.274932 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267326    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267328    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.267331    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.267880    2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.273900    2575 server.go:530] "Kubelet version" kubeletVersion="v1.33.12"
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.273916    2575 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273963    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273968    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273971    2575 feature_gate.go:328] unrecognized feature gate: Example
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273975    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273978    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273981    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273983    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273986    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273988    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273991    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 17:57:05.275426 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273993    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273996    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.273998    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274001    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274003    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274007    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274012    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274014    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274017    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274020    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274023    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274025    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274028    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274030    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274033    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274036    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274038    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274041    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274043    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274046    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 17:57:05.275832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274049    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274053    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274056    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274059    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274061    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274064    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274066    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274069    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274072    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274075    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274077    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274080    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274082    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274085    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274087    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274090    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274092    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274095    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274098    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274100    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 17:57:05.276389 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274103    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274105    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274108    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274110    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274112    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274115    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274118    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274120    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274123    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274125    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274128    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274130    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274133    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274137    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274140    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274143    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274148    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274169    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274174    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 17:57:05.276907 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274177    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274181    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274184    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274186    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274189    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274191    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274194    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274196    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274199    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274201    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274204    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274207    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274210    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274212    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274215    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274217    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 17:57:05.277386 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274220    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.274225    2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274323    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274327    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274330    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274333    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274336    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274339    2575 feature_gate.go:328] unrecognized feature gate: Example
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274342    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274345    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274348    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274351    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274364    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274367    2575 feature_gate.go:328] unrecognized feature gate: Example2
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274370    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274373    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 17:57:05.277783 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274376    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274378    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274381    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274383    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274386    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274388    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274391    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274393    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274396    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274398    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274401    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274403    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274406    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274409    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274411    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274414    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274416    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274419    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274422    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 17:57:05.278202 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274424    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274426    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274429    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274431    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274434    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274436    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274439    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274441    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274444    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274447    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274450    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274452    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274455    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274458    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274461    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274463    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274466    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274468    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274471    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274473    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 17:57:05.278695 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274476    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274478    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274481    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274483    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274486    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274488    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274490    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274493    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274495    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274498    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274501    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274505    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274507    2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274510    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274512    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274515    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274517    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274520    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274523    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 17:57:05.279206 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274526    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274529    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274531    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274534    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274537    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274539    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274542    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274544    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274547    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274549    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274552    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274554    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274557    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:05.274559    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.274564    2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 17:57:05.279690 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.274670    2575 server.go:962] "Client rotation is on, will bootstrap in background"
May 29 17:57:05.280106 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.278436    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 29 17:57:05.280106 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.279502    2575 server.go:1019] "Starting client certificate rotation"
May 29 17:57:05.280106 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.279601    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 29 17:57:05.280106 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.279640    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 29 17:57:05.305115 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.305093    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 29 17:57:05.306994 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.306976    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 29 17:57:05.320721 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.320702    2575 log.go:25] "Validated CRI v1 runtime API"
May 29 17:57:05.326787 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.326772    2575 log.go:25] "Validated CRI v1 image API"
May 29 17:57:05.330479 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.330460    2575 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 29 17:57:05.333325 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.333307    2575 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 29 17:57:05.335233 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.335212    2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b0f35555-48df-4b93-820f-679153e83c11:/dev/nvme0n1p3 cc7b47e4-fa1d-43b4-a4de-9aabedfd2305:/dev/nvme0n1p4]
May 29 17:57:05.335305 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.335233    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 29 17:57:05.339580 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.339471    2575 manager.go:217] Machine: {Timestamp:2026-05-29 17:57:05.338959088 +0000 UTC m=+0.394752441 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099486 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21fe011215571a2a6f500090abd719 SystemUUID:ec21fe01-1215-571a-2a6f-500090abd719 BootID:f963f3b0-5297-435d-aaec-eac63a6f82bd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a1:b0:15:ad:d5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a1:b0:15:ad:d5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:ea:40:01:59:ef Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 29 17:57:05.340944 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.340933    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 29 17:57:05.341029 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.341017    2575 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 29 17:57:05.343930 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.343901    2575 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 29 17:57:05.344088 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.343932    2575 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-167.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 29 17:57:05.344182 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.344101    2575 topology_manager.go:138] "Creating topology manager with none policy"
May 29 17:57:05.344182 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.344114    2575 container_manager_linux.go:306] "Creating device plugin manager"
May 29 17:57:05.344182 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.344125    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 29 17:57:05.344830 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.344817    2575 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 29 17:57:05.346896 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.346883    2575 state_mem.go:36] "Initialized new in-memory state store"
May 29 17:57:05.347029 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.347018    2575 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 29 17:57:05.348962 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.348951    2575 kubelet.go:491] "Attempting to sync node with API server"
May 29 17:57:05.349016 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.348967    2575 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 29 17:57:05.349016 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.348983    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 29 17:57:05.349016 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.348997    2575 kubelet.go:397] "Adding apiserver pod source"
May 29 17:57:05.349016 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.349009    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 29 17:57:05.350058 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.350045    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 29 17:57:05.350123 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.350068    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 29 17:57:05.352885 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.352874    2575 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1"
May 29 17:57:05.354495 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.354481    2575 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 29 17:57:05.356201 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356190    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 29 17:57:05.356242 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356207    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 29 17:57:05.356242 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356214    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 29 17:57:05.356242 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356220    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 29 17:57:05.356242 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356228    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 29 17:57:05.356242 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356234    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 29 17:57:05.356242 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356239    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 29 17:57:05.356402 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356251    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 29 17:57:05.356402 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356258    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 29 17:57:05.356402 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356264    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 29 17:57:05.356402 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356277    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 29 17:57:05.356402 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356283    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 29 17:57:05.356970 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356959    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 29 17:57:05.356970 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.356969    2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 29 17:57:05.358469 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.358443    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dt4sj"
May 29 17:57:05.359670 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.359640    2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-167.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 29 17:57:05.359761 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.359718    2575 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-167.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 29 17:57:05.359799 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.359776    2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 29 17:57:05.360961 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.360949    2575 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 29 17:57:05.361004 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.360986    2575 server.go:1295] "Started kubelet"
May 29 17:57:05.361070 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.361048    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 29 17:57:05.361134 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.361081    2575 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 29 17:57:05.361198 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.361184    2575 server_v1.go:47] "podresources" method="list" useActivePods=true
May 29 17:57:05.361902 ip-10-0-133-167 systemd[1]: Started Kubernetes Kubelet.
May 29 17:57:05.362508 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.362355    2575 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 29 17:57:05.362824 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.362814    2575 server.go:317] "Adding debug handlers to kubelet server"
May 29 17:57:05.366768 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.366731    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dt4sj"
May 29 17:57:05.367133 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.367116    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 29 17:57:05.367632 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.367267    2575 volume_manager.go:295] "The desired_state_of_world populator starts"
May 29 17:57:05.367632 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.367282    2575 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 29 17:57:05.367632 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.367342    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 29 17:57:05.367632 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.367432    2575 reconstruct.go:97] "Volume reconstruction finished"
May 29 17:57:05.367632 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.367443    2575 reconciler.go:26] "Reconciler: start to sync state"
May 29 17:57:05.367632 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.367399    2575 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 29 17:57:05.369933 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.369279    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:05.370110 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.370092    2575 factory.go:55] Registering systemd factory
May 29 17:57:05.370232 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.370221    2575 factory.go:223] Registration of the systemd container factory successfully
May 29 17:57:05.370990 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.370837    2575 factory.go:153] Registering CRI-O factory
May 29 17:57:05.371170 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.371141    2575 factory.go:223] Registration of the crio container factory successfully
May 29 17:57:05.371263 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.371249    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 29 17:57:05.371325 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.371269    2575 factory.go:103] Registering Raw factory
May 29 17:57:05.371325 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.371263    2575 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 29 17:57:05.371325 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.371284    2575 manager.go:1196] Started watching for new ooms in manager
May 29 17:57:05.371683 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.371670    2575 manager.go:319] Starting recovery of all containers
May 29 17:57:05.378453 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.378427    2575 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:05.381612 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.381594    2575 manager.go:324] Recovery completed
May 29 17:57:05.383982 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.383904    2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-167.ec2.internal\" not found" node="ip-10-0-133-167.ec2.internal"
May 29 17:57:05.385852 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.385840    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:05.388007 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.387992    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:05.388073 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.388019    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:05.388073 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.388029    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:05.388587 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.388570    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 29 17:57:05.388587 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.388584    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 29 17:57:05.388681 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.388600    2575 state_mem.go:36] "Initialized new in-memory state store"
May 29 17:57:05.390504 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.390489    2575 policy_none.go:49] "None policy: Start"
May 29 17:57:05.390504 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.390506    2575 memory_manager.go:186] "Starting memorymanager" policy="None"
May 29 17:57:05.390625 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.390516    2575 state_mem.go:35] "Initializing new in-memory state store"
May 29 17:57:05.433738 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.433722    2575 manager.go:341] "Starting Device Plugin manager"
May 29 17:57:05.435734 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.433753    2575 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 29 17:57:05.435734 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.433763    2575 server.go:85] "Starting device plugin registration server"
May 29 17:57:05.435734 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.434033    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
May 29 17:57:05.435734 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.434044    2575 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 29 17:57:05.435734 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.434201    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 29 17:57:05.435734 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.434268    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 29 17:57:05.435734 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.434276    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 29 17:57:05.435734 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.434760    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 29 17:57:05.435734 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.434805    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:05.462092 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.462049    2575 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 29 17:57:05.463290 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.463271    2575 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 29 17:57:05.463398 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.463294    2575 status_manager.go:230] "Starting to sync pod status with apiserver"
May 29 17:57:05.463398 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.463318    2575 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 29 17:57:05.463398 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.463327    2575 kubelet.go:2451] "Starting kubelet main sync loop"
May 29 17:57:05.463398 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.463374    2575 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 29 17:57:05.465042 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.465026    2575 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:05.535210 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.535137    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:05.536193 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.536175    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:05.536296 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.536207    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:05.536296 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.536223    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:05.536296 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.536251    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-167.ec2.internal"
May 29 17:57:05.545149 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.545133    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-167.ec2.internal"
May 29 17:57:05.545249 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.545172    2575 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-167.ec2.internal\": node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:05.564111 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.564090    2575 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal"]
May 29 17:57:05.564221 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.564171    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:05.564971 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.564957    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:05.565025 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.564986    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:05.565025 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.564999    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:05.568167 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.568139    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:05.568293 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.568279    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.568334 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.568306    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:05.568873 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.568855    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:05.568964 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.568862    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:05.568964 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.568914    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:05.568964 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.568932    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:05.568964 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.568882    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:05.569107 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.568988    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:05.569918 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.569903    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.569991 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.569927    2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 17:57:05.570718 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.570703    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientMemory"
May 29 17:57:05.570789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.570727    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasNoDiskPressure"
May 29 17:57:05.570789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.570736    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeHasSufficientPID"
May 29 17:57:05.571098 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.571082    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:05.604726 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.604709    2575 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-167.ec2.internal\" not found" node="ip-10-0-133-167.ec2.internal"
May 29 17:57:05.609047 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.609030    2575 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-167.ec2.internal\" not found" node="ip-10-0-133-167.ec2.internal"
May 29 17:57:05.668710 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.668691    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16b760888857fdc72e7a0e00f59262e9-config\") pod \"kube-apiserver-proxy-ip-10-0-133-167.ec2.internal\" (UID: \"16b760888857fdc72e7a0e00f59262e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.668783 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.668715    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/247e05a29ffc160e0f5cb65ed747c7c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal\" (UID: \"247e05a29ffc160e0f5cb65ed747c7c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.668783 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.668733    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/247e05a29ffc160e0f5cb65ed747c7c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal\" (UID: \"247e05a29ffc160e0f5cb65ed747c7c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.672199 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.672184    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:05.769009 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.768986    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16b760888857fdc72e7a0e00f59262e9-config\") pod \"kube-apiserver-proxy-ip-10-0-133-167.ec2.internal\" (UID: \"16b760888857fdc72e7a0e00f59262e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.769009 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.769003    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16b760888857fdc72e7a0e00f59262e9-config\") pod \"kube-apiserver-proxy-ip-10-0-133-167.ec2.internal\" (UID: \"16b760888857fdc72e7a0e00f59262e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.769186 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.769028    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/247e05a29ffc160e0f5cb65ed747c7c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal\" (UID: \"247e05a29ffc160e0f5cb65ed747c7c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.769186 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.769054    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/247e05a29ffc160e0f5cb65ed747c7c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal\" (UID: \"247e05a29ffc160e0f5cb65ed747c7c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.769186 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.769089    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/247e05a29ffc160e0f5cb65ed747c7c9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal\" (UID: \"247e05a29ffc160e0f5cb65ed747c7c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.769186 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.769091    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/247e05a29ffc160e0f5cb65ed747c7c9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal\" (UID: \"247e05a29ffc160e0f5cb65ed747c7c9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.773071 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.773054    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:05.873774 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.873746    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:05.906953 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.906931    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.912297 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:05.912278    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal"
May 29 17:57:05.974622 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:05.974589    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:06.075104 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:06.075071    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:06.175598 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:06.175524    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:06.276025 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:06.275996    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:06.279175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.279143    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 29 17:57:06.279321 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.279304    2575 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 17:57:06.279369 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.279326    2575 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 17:57:06.367556 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.367532    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 29 17:57:06.368574 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.368538    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-28 17:52:05 +0000 UTC" deadline="2028-01-19 20:04:59.725483938 +0000 UTC"
May 29 17:57:06.368574 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.368572    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14402h7m53.35691707s"
May 29 17:57:06.376765 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:06.376717    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:06.381037 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.381019    2575 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 29 17:57:06.411534 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.411510    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mlbpr"
May 29 17:57:06.419440 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.419421    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mlbpr"
May 29 17:57:06.422466 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:06.422427    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod247e05a29ffc160e0f5cb65ed747c7c9.slice/crio-49747df95502e9113df752f9203cba6b36a86b5e663fcd3de4917c4a15338827 WatchSource:0}: Error finding container 49747df95502e9113df752f9203cba6b36a86b5e663fcd3de4917c4a15338827: Status 404 returned error can't find the container with id 49747df95502e9113df752f9203cba6b36a86b5e663fcd3de4917c4a15338827
May 29 17:57:06.422653 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:06.422636    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b760888857fdc72e7a0e00f59262e9.slice/crio-5d7f670bda87c7b55864d58b2382528635c267373627e32efb3509dedc33a3c8 WatchSource:0}: Error finding container 5d7f670bda87c7b55864d58b2382528635c267373627e32efb3509dedc33a3c8: Status 404 returned error can't find the container with id 5d7f670bda87c7b55864d58b2382528635c267373627e32efb3509dedc33a3c8
May 29 17:57:06.425640 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.425591    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 17:57:06.466606 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.466566    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal" event={"ID":"247e05a29ffc160e0f5cb65ed747c7c9","Type":"ContainerStarted","Data":"49747df95502e9113df752f9203cba6b36a86b5e663fcd3de4917c4a15338827"}
May 29 17:57:06.467511 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.467490    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal" event={"ID":"16b760888857fdc72e7a0e00f59262e9","Type":"ContainerStarted","Data":"5d7f670bda87c7b55864d58b2382528635c267373627e32efb3509dedc33a3c8"}
May 29 17:57:06.477841 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:06.477822    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:06.578334 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:06.578308    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:06.678827 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:06.678759    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:06.779254 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:06.779220    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:06.880076 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:06.880041    2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-167.ec2.internal\" not found"
May 29 17:57:06.942241 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.942149    2575 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:06.955124 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.954982    2575 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:06.968551 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.968525    2575 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal"
May 29 17:57:06.980590 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.980564    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 29 17:57:06.981567 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.981548    2575 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal"
May 29 17:57:06.991078 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:06.991050    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 29 17:57:07.349780 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.349705    2575 apiserver.go:52] "Watching apiserver"
May 29 17:57:07.355684 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.355657    2575 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 29 17:57:07.356067 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.356039    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-4spp5","openshift-multus/multus-zctkf","openshift-network-diagnostics/network-check-target-cp2gr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal","openshift-multus/multus-additional-cni-plugins-94hsf","openshift-multus/network-metrics-daemon-zsvdq","openshift-network-operator/iptables-alerter-6jcqq","openshift-ovn-kubernetes/ovnkube-node-cxqf9","kube-system/konnectivity-agent-gcd7p","kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2","openshift-cluster-node-tuning-operator/tuned-28z92","openshift-dns/node-resolver-rq5ft"]
May 29 17:57:07.358723 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.358700    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.359542 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.359520    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zctkf"
May 29 17:57:07.360511 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.360489    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:07.360610 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:07.360566    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:07.361419 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.361401    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.361520 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.361502    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.362679 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.362312    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 29 17:57:07.362679 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.362394    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 29 17:57:07.362830 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.362784    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:07.362938 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.362910    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-48zrg\""
May 29 17:57:07.363124 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.363105    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 29 17:57:07.363394 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.363373    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 29 17:57:07.363567 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.363549    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s56qk\""
May 29 17:57:07.363636 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.363609    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 29 17:57:07.363813 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.363428    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 29 17:57:07.363891 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:07.363843    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:07.364305 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.364285    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 29 17:57:07.364639 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.364614    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 29 17:57:07.365463 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.365442    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 29 17:57:07.365546 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.365449    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 29 17:57:07.365546 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.365493    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 29 17:57:07.366497 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.366383    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z5dsf\""
May 29 17:57:07.366497 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.366428    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mfmxg\""
May 29 17:57:07.366643 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.366498    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.367728 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.367712    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.367820 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.367798    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:07.368981 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.368958    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.369345 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.369133    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 29 17:57:07.369672 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.369643    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 29 17:57:07.369766 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.369650    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tbj7j\""
May 29 17:57:07.369766 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.369652    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 29 17:57:07.370094 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.370073    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.371289 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.371272    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 29 17:57:07.371462 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.371444    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 29 17:57:07.372034 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.372020    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-clqgh\""
May 29 17:57:07.372183 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.372142    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b2dd9\""
May 29 17:57:07.372492 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.372477    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 29 17:57:07.373134 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.373118    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 29 17:57:07.373219 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.373119    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 29 17:57:07.373219 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.373169    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 29 17:57:07.373219 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.373187    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 29 17:57:07.373679 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.373664    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 29 17:57:07.373751 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.373707    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 29 17:57:07.374042 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.374027    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-j5tgx\""
May 29 17:57:07.374397 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.374380    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 29 17:57:07.374487 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.374407    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 29 17:57:07.374487 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.374390    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w465t\""
May 29 17:57:07.374776 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.374753    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 29 17:57:07.374776 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.374762    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 29 17:57:07.377681 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377661    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-var-lib-kubelet\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.377764 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zrsk\" (UniqueName: \"kubernetes.io/projected/f9ca71bf-3069-42a6-95d4-106519e08945-kube-api-access-6zrsk\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.377764 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377733    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb-iptables-alerter-script\") pod \"iptables-alerter-6jcqq\" (UID: \"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb\") " pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.377868 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377765    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb-host-slash\") pod \"iptables-alerter-6jcqq\" (UID: \"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb\") " pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.377868 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377813    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f9ca71bf-3069-42a6-95d4-106519e08945-env-overrides\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.377958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377873    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9v5t\" (UniqueName: \"kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t\") pod \"network-check-target-cp2gr\" (UID: \"50d7a3bd-52e4-4d5d-8d79-fe64008680ab\") " pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:07.377958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377895    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-os-release\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.377958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-node-log\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.377958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377937    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-run-ovn-kubernetes\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.378091 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377963    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg6bc\" (UniqueName: \"kubernetes.io/projected/56fb789d-bd9e-4a88-958d-e8216ab60549-kube-api-access-dg6bc\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.378091 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.377989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-kubernetes\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.378091 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-systemd\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.378091 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378035    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-776qb\" (UniqueName: \"kubernetes.io/projected/a054de51-319b-4ab1-b4ef-5f2d979ef630-kube-api-access-776qb\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.378091 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378061    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-multus-socket-dir-parent\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378095    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-multus-conf-dir\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378118    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-run-openvswitch\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.378264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378175    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56fb789d-bd9e-4a88-958d-e8216ab60549-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.378264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378202    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26bjl\" (UniqueName: \"kubernetes.io/projected/c21facb6-ed73-479a-93b2-19822a7c2620-kube-api-access-26bjl\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378225    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-run-systemd\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.378264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378248    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-modprobe-d\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.378464 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378272    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-host\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.378464 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378297    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-cnibin\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378464 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378322    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-run-netns\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378464 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378357    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-var-lib-cni-bin\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378464 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378381    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-etc-kubernetes\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378464 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378426    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-etc-openvswitch\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.378464 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378450    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-run-ovn\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.378704 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378490    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0a29d4c2-23f1-4323-997a-1f5894d38144-agent-certs\") pod \"konnectivity-agent-gcd7p\" (UID: \"0a29d4c2-23f1-4323-997a-1f5894d38144\") " pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:07.378704 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0a29d4c2-23f1-4323-997a-1f5894d38144-konnectivity-ca\") pod \"konnectivity-agent-gcd7p\" (UID: \"0a29d4c2-23f1-4323-997a-1f5894d38144\") " pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:07.378704 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378558    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-log-socket\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.378704 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378581    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-sys\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.378704 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378602    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a054de51-319b-4ab1-b4ef-5f2d979ef630-tmp\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.378704 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378625    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-var-lib-cni-multus\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378704 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378679    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-sysctl-conf\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.378958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378713    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-hostroot\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378740    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9ca71bf-3069-42a6-95d4-106519e08945-ovn-node-metrics-cert\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.378958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378777    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-lib-modules\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.378958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378801    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-run\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.378958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378817    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-cnibin\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.378958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378837    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c21facb6-ed73-479a-93b2-19822a7c2620-cni-binary-copy\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378853    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-run-multus-certs\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.378958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gmvb\" (UniqueName: \"kubernetes.io/projected/19a8c87f-2157-4411-8e5a-b272ce1259f5-kube-api-access-5gmvb\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:07.378958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378929    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-var-lib-openvswitch\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.378958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378950    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-cni-netd\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378965    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.378983    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-system-cni-dir\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379005    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-sysctl-d\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379042    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-var-lib-kubelet\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379070    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379096    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b3a52ac-9cde-4e35-b89c-f5ff5b50093c-hosts-file\") pod \"node-resolver-rq5ft\" (UID: \"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c\") " pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379115    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c21facb6-ed73-479a-93b2-19822a7c2620-multus-daemon-config\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379129    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-systemd-units\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379144    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56fb789d-bd9e-4a88-958d-e8216ab60549-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379179    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5sqr\" (UniqueName: \"kubernetes.io/projected/2b3a52ac-9cde-4e35-b89c-f5ff5b50093c-kube-api-access-x5sqr\") pod \"node-resolver-rq5ft\" (UID: \"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c\") " pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-tuned\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379218    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-system-cni-dir\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379242    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-kubelet\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379263    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-sysconfig\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379280    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-multus-cni-dir\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.379436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379316    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-cni-bin\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379335    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-tuning-conf-dir\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379350    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29z69\" (UniqueName: \"kubernetes.io/projected/2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb-kube-api-access-29z69\") pod \"iptables-alerter-6jcqq\" (UID: \"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb\") " pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379365    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-slash\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-os-release\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379438    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56fb789d-bd9e-4a88-958d-e8216ab60549-cni-binary-copy\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379453    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-run-k8s-cni-cncf-io\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379477    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4578642-65b0-478f-8f73-afc1f47c3009-host\") pod \"node-ca-4spp5\" (UID: \"b4578642-65b0-478f-8f73-afc1f47c3009\") " pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379491    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4578642-65b0-478f-8f73-afc1f47c3009-serviceca\") pod \"node-ca-4spp5\" (UID: \"b4578642-65b0-478f-8f73-afc1f47c3009\") " pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379505    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75hr6\" (UniqueName: \"kubernetes.io/projected/b4578642-65b0-478f-8f73-afc1f47c3009-kube-api-access-75hr6\") pod \"node-ca-4spp5\" (UID: \"b4578642-65b0-478f-8f73-afc1f47c3009\") " pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379521    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-run-netns\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f9ca71bf-3069-42a6-95d4-106519e08945-ovnkube-config\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f9ca71bf-3069-42a6-95d4-106519e08945-ovnkube-script-lib\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.380175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.379597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3a52ac-9cde-4e35-b89c-f5ff5b50093c-tmp-dir\") pod \"node-resolver-rq5ft\" (UID: \"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c\") " pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.420759 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.420728    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 17:52:06 +0000 UTC" deadline="2027-11-13 01:27:15.537918667 +0000 UTC"
May 29 17:57:07.420759 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.420757    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12775h30m8.117165055s"
May 29 17:57:07.468553 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.468531    2575 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 29 17:57:07.480273 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480244    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs47c\" (UniqueName: \"kubernetes.io/projected/9a355e8a-109a-4751-9421-a27eb4a47c78-kube-api-access-fs47c\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.480441 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480286    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-29z69\" (UniqueName: \"kubernetes.io/projected/2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb-kube-api-access-29z69\") pod \"iptables-alerter-6jcqq\" (UID: \"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb\") " pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.480441 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480313    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-slash\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.480441 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480337    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-os-release\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.480441 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480364    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56fb789d-bd9e-4a88-958d-e8216ab60549-cni-binary-copy\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.480441 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480389    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-run-k8s-cni-cncf-io\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.480441 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480414    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4578642-65b0-478f-8f73-afc1f47c3009-host\") pod \"node-ca-4spp5\" (UID: \"b4578642-65b0-478f-8f73-afc1f47c3009\") " pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.480441 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480438    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4578642-65b0-478f-8f73-afc1f47c3009-serviceca\") pod \"node-ca-4spp5\" (UID: \"b4578642-65b0-478f-8f73-afc1f47c3009\") " pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480461    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-75hr6\" (UniqueName: \"kubernetes.io/projected/b4578642-65b0-478f-8f73-afc1f47c3009-kube-api-access-75hr6\") pod \"node-ca-4spp5\" (UID: \"b4578642-65b0-478f-8f73-afc1f47c3009\") " pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480460    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-os-release\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480470    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-slash\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480489    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-run-netns\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480522    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4578642-65b0-478f-8f73-afc1f47c3009-host\") pod \"node-ca-4spp5\" (UID: \"b4578642-65b0-478f-8f73-afc1f47c3009\") " pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480526    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f9ca71bf-3069-42a6-95d4-106519e08945-ovnkube-config\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480538    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-run-netns\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480563    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f9ca71bf-3069-42a6-95d4-106519e08945-ovnkube-script-lib\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480589    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3a52ac-9cde-4e35-b89c-f5ff5b50093c-tmp-dir\") pod \"node-resolver-rq5ft\" (UID: \"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c\") " pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480615    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-var-lib-kubelet\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480623    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-run-k8s-cni-cncf-io\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480640    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6zrsk\" (UniqueName: \"kubernetes.io/projected/f9ca71bf-3069-42a6-95d4-106519e08945-kube-api-access-6zrsk\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480665    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb-iptables-alerter-script\") pod \"iptables-alerter-6jcqq\" (UID: \"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb\") " pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480692    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb-host-slash\") pod \"iptables-alerter-6jcqq\" (UID: \"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb\") " pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480715    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f9ca71bf-3069-42a6-95d4-106519e08945-env-overrides\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480739    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j9v5t\" (UniqueName: \"kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t\") pod \"network-check-target-cp2gr\" (UID: \"50d7a3bd-52e4-4d5d-8d79-fe64008680ab\") " pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:07.480789 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480762    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-os-release\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480784    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-node-log\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480810    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-run-ovn-kubernetes\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480834    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dg6bc\" (UniqueName: \"kubernetes.io/projected/56fb789d-bd9e-4a88-958d-e8216ab60549-kube-api-access-dg6bc\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480861    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-kubernetes\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480884    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-systemd\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480910    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-776qb\" (UniqueName: \"kubernetes.io/projected/a054de51-319b-4ab1-b4ef-5f2d979ef630-kube-api-access-776qb\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480911    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56fb789d-bd9e-4a88-958d-e8216ab60549-cni-binary-copy\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480938    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-multus-socket-dir-parent\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480952    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3a52ac-9cde-4e35-b89c-f5ff5b50093c-tmp-dir\") pod \"node-resolver-rq5ft\" (UID: \"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c\") " pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480967    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-multus-conf-dir\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480997    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f9ca71bf-3069-42a6-95d4-106519e08945-ovnkube-config\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481003    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4578642-65b0-478f-8f73-afc1f47c3009-serviceca\") pod \"node-ca-4spp5\" (UID: \"b4578642-65b0-478f-8f73-afc1f47c3009\") " pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.480991    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-run-openvswitch\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481021    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-node-log\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481041    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56fb789d-bd9e-4a88-958d-e8216ab60549-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481080    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-26bjl\" (UniqueName: \"kubernetes.io/projected/c21facb6-ed73-479a-93b2-19822a7c2620-kube-api-access-26bjl\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.481609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481084    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-var-lib-kubelet\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481106    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-run-systemd\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481130    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-modprobe-d\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481176    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-host\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481202    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-cnibin\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481225    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-run-netns\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481254    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-var-lib-cni-bin\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481279    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-etc-kubernetes\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481308    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-etc-openvswitch\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481328    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb-host-slash\") pod \"iptables-alerter-6jcqq\" (UID: \"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb\") " pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481349    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-run-ovn\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481361    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-run-ovn-kubernetes\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481410    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb-iptables-alerter-script\") pod \"iptables-alerter-6jcqq\" (UID: \"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb\") " pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481413    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0a29d4c2-23f1-4323-997a-1f5894d38144-agent-certs\") pod \"konnectivity-agent-gcd7p\" (UID: \"0a29d4c2-23f1-4323-997a-1f5894d38144\") " pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481424    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f9ca71bf-3069-42a6-95d4-106519e08945-env-overrides\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481427    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-run-systemd\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481324    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f9ca71bf-3069-42a6-95d4-106519e08945-ovnkube-script-lib\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481448    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-kubernetes\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.482493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481487    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-systemd\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481514    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-etc-openvswitch\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481553    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-os-release\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481583    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-host\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481600    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-multus-conf-dir\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481644    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-etc-kubernetes\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481651    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-multus-socket-dir-parent\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481671    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-cnibin\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481688    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-run-openvswitch\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481725    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-run-ovn\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481649    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-modprobe-d\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481770    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0a29d4c2-23f1-4323-997a-1f5894d38144-konnectivity-ca\") pod \"konnectivity-agent-gcd7p\" (UID: \"0a29d4c2-23f1-4323-997a-1f5894d38144\") " pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481790    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-run-netns\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481806    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56fb789d-bd9e-4a88-958d-e8216ab60549-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481849    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481894    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-log-socket\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481966    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-sys\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.481993    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a054de51-319b-4ab1-b4ef-5f2d979ef630-tmp\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482040    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-var-lib-cni-multus\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.483328 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482087    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-sys\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482149    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-sysctl-conf\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482146    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-var-lib-cni-multus\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482177    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-var-lib-cni-bin\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482199    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-log-socket\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482245    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-hostroot\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482289    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-socket-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482299    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-hostroot\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482339    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9ca71bf-3069-42a6-95d4-106519e08945-ovn-node-metrics-cert\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482367    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-lib-modules\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482368    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0a29d4c2-23f1-4323-997a-1f5894d38144-konnectivity-ca\") pod \"konnectivity-agent-gcd7p\" (UID: \"0a29d4c2-23f1-4323-997a-1f5894d38144\") " pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482388    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-sysctl-conf\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482404    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-run\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482449    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-cnibin\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482487    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-run\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482491    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-lib-modules\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482526    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c21facb6-ed73-479a-93b2-19822a7c2620-cni-binary-copy\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482550    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-run-multus-certs\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.484304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482529    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-cnibin\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482581    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482639    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-run-multus-certs\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482653    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5gmvb\" (UniqueName: \"kubernetes.io/projected/19a8c87f-2157-4411-8e5a-b272ce1259f5-kube-api-access-5gmvb\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482676    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-var-lib-openvswitch\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482702    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-cni-netd\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482752    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-var-lib-openvswitch\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482763    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482808    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-cni-netd\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482808    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-system-cni-dir\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482854    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-system-cni-dir\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482886    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-sysctl-d\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482904    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482913    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-var-lib-kubelet\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482952    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-etc-selinux\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482976    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482980    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-host-var-lib-kubelet\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.485121 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.482998    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b3a52ac-9cde-4e35-b89c-f5ff5b50093c-hosts-file\") pod \"node-resolver-rq5ft\" (UID: \"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c\") " pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483007    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-sysctl-d\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483041    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c21facb6-ed73-479a-93b2-19822a7c2620-multus-daemon-config\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-sys-fs\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483078    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b3a52ac-9cde-4e35-b89c-f5ff5b50093c-hosts-file\") pod \"node-resolver-rq5ft\" (UID: \"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c\") " pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483092    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-systemd-units\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:07.483095    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483118    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56fb789d-bd9e-4a88-958d-e8216ab60549-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483145    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x5sqr\" (UniqueName: \"kubernetes.io/projected/2b3a52ac-9cde-4e35-b89c-f5ff5b50093c-kube-api-access-x5sqr\") pod \"node-resolver-rq5ft\" (UID: \"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c\") " pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483188    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-tuned\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:07.483206    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs podName:19a8c87f-2157-4411-8e5a-b272ce1259f5 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:07.983185447 +0000 UTC m=+3.038978811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs") pod "network-metrics-daemon-zsvdq" (UID: "19a8c87f-2157-4411-8e5a-b272ce1259f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483294    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-systemd-units\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483550    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c21facb6-ed73-479a-93b2-19822a7c2620-multus-daemon-config\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483709    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-system-cni-dir\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483835    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c21facb6-ed73-479a-93b2-19822a7c2620-cni-binary-copy\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483883    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-system-cni-dir\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483925    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-registration-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.485947 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.483988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-device-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484016    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-kubelet\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484054    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-sysconfig\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484077    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-multus-cni-dir\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484115    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-cni-bin\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484138    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-tuning-conf-dir\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484191    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-kubelet\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484270    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c21facb6-ed73-479a-93b2-19822a7c2620-multus-cni-dir\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484319    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-sysconfig\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484345    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56fb789d-bd9e-4a88-958d-e8216ab60549-tuning-conf-dir\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484359    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9ca71bf-3069-42a6-95d4-106519e08945-host-cni-bin\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.484799    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56fb789d-bd9e-4a88-958d-e8216ab60549-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.485562    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0a29d4c2-23f1-4323-997a-1f5894d38144-agent-certs\") pod \"konnectivity-agent-gcd7p\" (UID: \"0a29d4c2-23f1-4323-997a-1f5894d38144\") " pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.485775    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a054de51-319b-4ab1-b4ef-5f2d979ef630-tmp\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.485993    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f9ca71bf-3069-42a6-95d4-106519e08945-ovn-node-metrics-cert\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.486608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.486051    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a054de51-319b-4ab1-b4ef-5f2d979ef630-etc-tuned\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.489514 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:07.489211    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:07.489514 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:07.489233    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:07.489514 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:07.489244    2575 projected.go:194] Error preparing data for projected volume kube-api-access-j9v5t for pod openshift-network-diagnostics/network-check-target-cp2gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:07.489514 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:07.489293    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t podName:50d7a3bd-52e4-4d5d-8d79-fe64008680ab nodeName:}" failed. No retries permitted until 2026-05-29 17:57:07.989276012 +0000 UTC m=+3.045069354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j9v5t" (UniqueName: "kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t") pod "network-check-target-cp2gr" (UID: "50d7a3bd-52e4-4d5d-8d79-fe64008680ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:07.491820 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.491779    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29z69\" (UniqueName: \"kubernetes.io/projected/2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb-kube-api-access-29z69\") pod \"iptables-alerter-6jcqq\" (UID: \"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb\") " pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.492293 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.492007    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75hr6\" (UniqueName: \"kubernetes.io/projected/b4578642-65b0-478f-8f73-afc1f47c3009-kube-api-access-75hr6\") pod \"node-ca-4spp5\" (UID: \"b4578642-65b0-478f-8f73-afc1f47c3009\") " pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.492802 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.492771    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zrsk\" (UniqueName: \"kubernetes.io/projected/f9ca71bf-3069-42a6-95d4-106519e08945-kube-api-access-6zrsk\") pod \"ovnkube-node-cxqf9\" (UID: \"f9ca71bf-3069-42a6-95d4-106519e08945\") " pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.493223 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.493179    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-776qb\" (UniqueName: \"kubernetes.io/projected/a054de51-319b-4ab1-b4ef-5f2d979ef630-kube-api-access-776qb\") pod \"tuned-28z92\" (UID: \"a054de51-319b-4ab1-b4ef-5f2d979ef630\") " pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.494037 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.494014    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gmvb\" (UniqueName: \"kubernetes.io/projected/19a8c87f-2157-4411-8e5a-b272ce1259f5-kube-api-access-5gmvb\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:07.494320 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.494257    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26bjl\" (UniqueName: \"kubernetes.io/projected/c21facb6-ed73-479a-93b2-19822a7c2620-kube-api-access-26bjl\") pod \"multus-zctkf\" (UID: \"c21facb6-ed73-479a-93b2-19822a7c2620\") " pod="openshift-multus/multus-zctkf"
May 29 17:57:07.494320 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.494302    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5sqr\" (UniqueName: \"kubernetes.io/projected/2b3a52ac-9cde-4e35-b89c-f5ff5b50093c-kube-api-access-x5sqr\") pod \"node-resolver-rq5ft\" (UID: \"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c\") " pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.494779 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.494722    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg6bc\" (UniqueName: \"kubernetes.io/projected/56fb789d-bd9e-4a88-958d-e8216ab60549-kube-api-access-dg6bc\") pod \"multus-additional-cni-plugins-94hsf\" (UID: \"56fb789d-bd9e-4a88-958d-e8216ab60549\") " pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.584764 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584723    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-socket-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.584936 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584779    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.584936 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584807    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-etc-selinux\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.584936 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584840    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-sys-fs\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.584936 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584867    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-registration-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.584936 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584867    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-socket-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.584936 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584916    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-sys-fs\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.584936 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584918    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.584936 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584932    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-registration-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.585292 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584929    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-etc-selinux\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.585292 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584940    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-device-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.585292 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584975    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fs47c\" (UniqueName: \"kubernetes.io/projected/9a355e8a-109a-4751-9421-a27eb4a47c78-kube-api-access-fs47c\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.585292 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.584999    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a355e8a-109a-4751-9421-a27eb4a47c78-device-dir\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.594746 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.594726    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs47c\" (UniqueName: \"kubernetes.io/projected/9a355e8a-109a-4751-9421-a27eb4a47c78-kube-api-access-fs47c\") pod \"aws-ebs-csi-driver-node-6vjk2\" (UID: \"9a355e8a-109a-4751-9421-a27eb4a47c78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.597516 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.597495    2575 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:07.672016 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.671970    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-28z92"
May 29 17:57:07.677930 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.677905    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zctkf"
May 29 17:57:07.686170 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.686133    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4spp5"
May 29 17:57:07.692749 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.692723    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-94hsf"
May 29 17:57:07.699522 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.699502    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6jcqq"
May 29 17:57:07.707061 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.707043    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:07.715589 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.715570    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:07.722225 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.722204    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rq5ft"
May 29 17:57:07.722419 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.722399    2575 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 17:57:07.727750 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.727734    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2"
May 29 17:57:07.882355 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:07.882328    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56fb789d_bd9e_4a88_958d_e8216ab60549.slice/crio-e5baeea8d57e88913247211f0ec039a624781b09fc0d166824bd8806cec32020 WatchSource:0}: Error finding container e5baeea8d57e88913247211f0ec039a624781b09fc0d166824bd8806cec32020: Status 404 returned error can't find the container with id e5baeea8d57e88913247211f0ec039a624781b09fc0d166824bd8806cec32020
May 29 17:57:07.884019 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:07.883998    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9ca71bf_3069_42a6_95d4_106519e08945.slice/crio-5b8425b9ac8391a5c82eab08a1051068747ddcb316953d7332d47b4e82349915 WatchSource:0}: Error finding container 5b8425b9ac8391a5c82eab08a1051068747ddcb316953d7332d47b4e82349915: Status 404 returned error can't find the container with id 5b8425b9ac8391a5c82eab08a1051068747ddcb316953d7332d47b4e82349915
May 29 17:57:07.885195 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:07.885097    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda054de51_319b_4ab1_b4ef_5f2d979ef630.slice/crio-5f51ea0700cad66585f0e9e4a2ffc2e0f98b925e9d942a3c33ae6589db420106 WatchSource:0}: Error finding container 5f51ea0700cad66585f0e9e4a2ffc2e0f98b925e9d942a3c33ae6589db420106: Status 404 returned error can't find the container with id 5f51ea0700cad66585f0e9e4a2ffc2e0f98b925e9d942a3c33ae6589db420106
May 29 17:57:07.887672 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:07.887648    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eeb97a9_d522_47bc_ba71_c4fc2fc9a4eb.slice/crio-70ce639de5cb1cbaff136fe0626d4e91d962611fc5eee35a87a5583ad7dda7d5 WatchSource:0}: Error finding container 70ce639de5cb1cbaff136fe0626d4e91d962611fc5eee35a87a5583ad7dda7d5: Status 404 returned error can't find the container with id 70ce639de5cb1cbaff136fe0626d4e91d962611fc5eee35a87a5583ad7dda7d5
May 29 17:57:07.888566 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:07.888542    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a355e8a_109a_4751_9421_a27eb4a47c78.slice/crio-eb50c96e77de1cd5449ec9c333d1930fd221672da64ea1f1ed4ace3aa594013d WatchSource:0}: Error finding container eb50c96e77de1cd5449ec9c333d1930fd221672da64ea1f1ed4ace3aa594013d: Status 404 returned error can't find the container with id eb50c96e77de1cd5449ec9c333d1930fd221672da64ea1f1ed4ace3aa594013d
May 29 17:57:07.889338 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:07.889269    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a29d4c2_23f1_4323_997a_1f5894d38144.slice/crio-897e706267b6813a48099862aeabcb395e0cd4d11e20387039e73dcccbb4e1c9 WatchSource:0}: Error finding container 897e706267b6813a48099862aeabcb395e0cd4d11e20387039e73dcccbb4e1c9: Status 404 returned error can't find the container with id 897e706267b6813a48099862aeabcb395e0cd4d11e20387039e73dcccbb4e1c9
May 29 17:57:07.890255 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:07.890205    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3a52ac_9cde_4e35_b89c_f5ff5b50093c.slice/crio-4e9caab1be5b31909bd68a373094b500effc65f5674ad00fd179a56376017c9f WatchSource:0}: Error finding container 4e9caab1be5b31909bd68a373094b500effc65f5674ad00fd179a56376017c9f: Status 404 returned error can't find the container with id 4e9caab1be5b31909bd68a373094b500effc65f5674ad00fd179a56376017c9f
May 29 17:57:07.892670 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:07.892511    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc21facb6_ed73_479a_93b2_19822a7c2620.slice/crio-34905d85827f6e5f8bddecfaa8ed9febd9d6f6030c213532c61a6822cc20e5a1 WatchSource:0}: Error finding container 34905d85827f6e5f8bddecfaa8ed9febd9d6f6030c213532c61a6822cc20e5a1: Status 404 returned error can't find the container with id 34905d85827f6e5f8bddecfaa8ed9febd9d6f6030c213532c61a6822cc20e5a1
May 29 17:57:07.893108 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:57:07.893085    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4578642_65b0_478f_8f73_afc1f47c3009.slice/crio-cf50a551f1ca83a73dba1ad7e97a6ff1d6b12f801133e0b5c222a626f0be49b3 WatchSource:0}: Error finding container cf50a551f1ca83a73dba1ad7e97a6ff1d6b12f801133e0b5c222a626f0be49b3: Status 404 returned error can't find the container with id cf50a551f1ca83a73dba1ad7e97a6ff1d6b12f801133e0b5c222a626f0be49b3
May 29 17:57:07.988205 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:07.988019    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:07.988329 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:07.988179    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:07.988386 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:07.988355    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs podName:19a8c87f-2157-4411-8e5a-b272ce1259f5 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:08.988332429 +0000 UTC m=+4.044125780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs") pod "network-metrics-daemon-zsvdq" (UID: "19a8c87f-2157-4411-8e5a-b272ce1259f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:08.088857 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.088832    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j9v5t\" (UniqueName: \"kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t\") pod \"network-check-target-cp2gr\" (UID: \"50d7a3bd-52e4-4d5d-8d79-fe64008680ab\") " pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:08.089010 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:08.088991    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:08.089051 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:08.089014    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:08.089051 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:08.089026    2575 projected.go:194] Error preparing data for projected volume kube-api-access-j9v5t for pod openshift-network-diagnostics/network-check-target-cp2gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:08.089119 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:08.089081    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t podName:50d7a3bd-52e4-4d5d-8d79-fe64008680ab nodeName:}" failed. No retries permitted until 2026-05-29 17:57:09.089063458 +0000 UTC m=+4.144856798 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9v5t" (UniqueName: "kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t") pod "network-check-target-cp2gr" (UID: "50d7a3bd-52e4-4d5d-8d79-fe64008680ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:08.422054 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.422003    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 17:52:06 +0000 UTC" deadline="2028-01-28 09:37:57.582296992 +0000 UTC"
May 29 17:57:08.422054 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.422041    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14607h40m49.160259402s"
May 29 17:57:08.464970 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.464488    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:08.464970 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:08.464610    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:08.482623 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.482586    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rq5ft" event={"ID":"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c","Type":"ContainerStarted","Data":"4e9caab1be5b31909bd68a373094b500effc65f5674ad00fd179a56376017c9f"}
May 29 17:57:08.505376 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.505340    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gcd7p" event={"ID":"0a29d4c2-23f1-4323-997a-1f5894d38144","Type":"ContainerStarted","Data":"897e706267b6813a48099862aeabcb395e0cd4d11e20387039e73dcccbb4e1c9"}
May 29 17:57:08.519294 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.519264    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2" event={"ID":"9a355e8a-109a-4751-9421-a27eb4a47c78","Type":"ContainerStarted","Data":"eb50c96e77de1cd5449ec9c333d1930fd221672da64ea1f1ed4ace3aa594013d"}
May 29 17:57:08.540409 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.540376    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6jcqq" event={"ID":"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb","Type":"ContainerStarted","Data":"70ce639de5cb1cbaff136fe0626d4e91d962611fc5eee35a87a5583ad7dda7d5"}
May 29 17:57:08.545839 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.545809    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-28z92" event={"ID":"a054de51-319b-4ab1-b4ef-5f2d979ef630","Type":"ContainerStarted","Data":"5f51ea0700cad66585f0e9e4a2ffc2e0f98b925e9d942a3c33ae6589db420106"}
May 29 17:57:08.549359 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.549323    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" event={"ID":"f9ca71bf-3069-42a6-95d4-106519e08945","Type":"ContainerStarted","Data":"5b8425b9ac8391a5c82eab08a1051068747ddcb316953d7332d47b4e82349915"}
May 29 17:57:08.555756 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.555729    2575 generic.go:358] "Generic (PLEG): container finished" podID="247e05a29ffc160e0f5cb65ed747c7c9" containerID="f79f931742fb703a9a993d88210a981f7dfc5c2ecef6df5c5917680db9321370" exitCode=0
May 29 17:57:08.555913 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.555890    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal" event={"ID":"247e05a29ffc160e0f5cb65ed747c7c9","Type":"ContainerDied","Data":"f79f931742fb703a9a993d88210a981f7dfc5c2ecef6df5c5917680db9321370"}
May 29 17:57:08.561624 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.561573    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4spp5" event={"ID":"b4578642-65b0-478f-8f73-afc1f47c3009","Type":"ContainerStarted","Data":"cf50a551f1ca83a73dba1ad7e97a6ff1d6b12f801133e0b5c222a626f0be49b3"}
May 29 17:57:08.572714 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.572686    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94hsf" event={"ID":"56fb789d-bd9e-4a88-958d-e8216ab60549","Type":"ContainerStarted","Data":"e5baeea8d57e88913247211f0ec039a624781b09fc0d166824bd8806cec32020"}
May 29 17:57:08.579387 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.579351    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal" event={"ID":"16b760888857fdc72e7a0e00f59262e9","Type":"ContainerStarted","Data":"2b0a9caf3a93d6626a08abcd3857f57da27dcc7939697bf139e18ba1e6c3dfd7"}
May 29 17:57:08.591416 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.591389    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zctkf" event={"ID":"c21facb6-ed73-479a-93b2-19822a7c2620","Type":"ContainerStarted","Data":"34905d85827f6e5f8bddecfaa8ed9febd9d6f6030c213532c61a6822cc20e5a1"}
May 29 17:57:08.598369 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:08.598321    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-167.ec2.internal" podStartSLOduration=2.598306227 podStartE2EDuration="2.598306227s" podCreationTimestamp="2026-05-29 17:57:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:57:08.598110767 +0000 UTC m=+3.653904133" watchObservedRunningTime="2026-05-29 17:57:08.598306227 +0000 UTC m=+3.654099590"
May 29 17:57:09.014843 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:09.014754    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:09.015008 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:09.014954    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:09.015065 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:09.015019    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs podName:19a8c87f-2157-4411-8e5a-b272ce1259f5 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:11.015000034 +0000 UTC m=+6.070793380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs") pod "network-metrics-daemon-zsvdq" (UID: "19a8c87f-2157-4411-8e5a-b272ce1259f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:09.115958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:09.115918    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j9v5t\" (UniqueName: \"kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t\") pod \"network-check-target-cp2gr\" (UID: \"50d7a3bd-52e4-4d5d-8d79-fe64008680ab\") " pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:09.116139 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:09.116128    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:09.116258 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:09.116148    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:09.116258 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:09.116175    2575 projected.go:194] Error preparing data for projected volume kube-api-access-j9v5t for pod openshift-network-diagnostics/network-check-target-cp2gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:09.116258 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:09.116236    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t podName:50d7a3bd-52e4-4d5d-8d79-fe64008680ab nodeName:}" failed. No retries permitted until 2026-05-29 17:57:11.116217703 +0000 UTC m=+6.172011044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9v5t" (UniqueName: "kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t") pod "network-check-target-cp2gr" (UID: "50d7a3bd-52e4-4d5d-8d79-fe64008680ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:09.464967 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:09.464464    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:09.464967 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:09.464609    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:09.603125 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:09.603091    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal" event={"ID":"247e05a29ffc160e0f5cb65ed747c7c9","Type":"ContainerStarted","Data":"ff3fdd363b8dbb97f0a72c8fd44fe8ccb41048cdcd2737ad643de82357081499"}
May 29 17:57:10.463860 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:10.463827    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:10.464055 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:10.463955    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:11.031687 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:11.031649    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:11.032140 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:11.031844    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:11.032140 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:11.031908    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs podName:19a8c87f-2157-4411-8e5a-b272ce1259f5 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:15.031888836 +0000 UTC m=+10.087682179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs") pod "network-metrics-daemon-zsvdq" (UID: "19a8c87f-2157-4411-8e5a-b272ce1259f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:11.132582 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:11.132495    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j9v5t\" (UniqueName: \"kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t\") pod \"network-check-target-cp2gr\" (UID: \"50d7a3bd-52e4-4d5d-8d79-fe64008680ab\") " pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:11.132760 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:11.132693    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:11.132760 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:11.132713    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:11.132760 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:11.132725    2575 projected.go:194] Error preparing data for projected volume kube-api-access-j9v5t for pod openshift-network-diagnostics/network-check-target-cp2gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:11.132931 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:11.132782    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t podName:50d7a3bd-52e4-4d5d-8d79-fe64008680ab nodeName:}" failed. No retries permitted until 2026-05-29 17:57:15.132763183 +0000 UTC m=+10.188556523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9v5t" (UniqueName: "kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t") pod "network-check-target-cp2gr" (UID: "50d7a3bd-52e4-4d5d-8d79-fe64008680ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:11.465319 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:11.464585    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:11.465319 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:11.464737    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:12.464806 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:12.464295    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:12.464806 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:12.464431    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:13.467701 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:13.467671    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:13.468146 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:13.467805    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:14.464468 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:14.463938    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:14.464468 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:14.464070    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:15.064773 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:15.064700    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:15.065258 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:15.064909    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:15.065258 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:15.064991    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs podName:19a8c87f-2157-4411-8e5a-b272ce1259f5 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:23.064969666 +0000 UTC m=+18.120763009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs") pod "network-metrics-daemon-zsvdq" (UID: "19a8c87f-2157-4411-8e5a-b272ce1259f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:15.165395 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:15.165305    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j9v5t\" (UniqueName: \"kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t\") pod \"network-check-target-cp2gr\" (UID: \"50d7a3bd-52e4-4d5d-8d79-fe64008680ab\") " pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:15.165581 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:15.165454    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:15.165581 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:15.165479    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:15.165581 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:15.165492    2575 projected.go:194] Error preparing data for projected volume kube-api-access-j9v5t for pod openshift-network-diagnostics/network-check-target-cp2gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:15.165581 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:15.165555    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t podName:50d7a3bd-52e4-4d5d-8d79-fe64008680ab nodeName:}" failed. No retries permitted until 2026-05-29 17:57:23.165535934 +0000 UTC m=+18.221329279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9v5t" (UniqueName: "kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t") pod "network-check-target-cp2gr" (UID: "50d7a3bd-52e4-4d5d-8d79-fe64008680ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:15.464974 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:15.464935    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:15.465171 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:15.465059    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:16.464512 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:16.464452    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:16.464968 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:16.464606    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:17.464134 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:17.464100    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:17.464333 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:17.464255    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:18.464408 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:18.464371    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:18.464793 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:18.464490    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:19.463868 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:19.463811    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:19.464047 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:19.463992    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:20.464354 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:20.464322    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:20.464786 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:20.464461    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:21.464345 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:21.464303    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:21.464495 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:21.464452    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:22.464013 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:22.463982    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:22.464222 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:22.464076    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:23.121649 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:23.121614    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:23.122073 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:23.121765    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:23.122073 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:23.121837    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs podName:19a8c87f-2157-4411-8e5a-b272ce1259f5 nodeName:}" failed. No retries permitted until 2026-05-29 17:57:39.121815762 +0000 UTC m=+34.177609107 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs") pod "network-metrics-daemon-zsvdq" (UID: "19a8c87f-2157-4411-8e5a-b272ce1259f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:23.222378 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:23.222341    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j9v5t\" (UniqueName: \"kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t\") pod \"network-check-target-cp2gr\" (UID: \"50d7a3bd-52e4-4d5d-8d79-fe64008680ab\") " pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:23.222559 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:23.222483    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:23.222559 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:23.222498    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:23.222559 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:23.222506    2575 projected.go:194] Error preparing data for projected volume kube-api-access-j9v5t for pod openshift-network-diagnostics/network-check-target-cp2gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:23.222559 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:23.222558    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t podName:50d7a3bd-52e4-4d5d-8d79-fe64008680ab nodeName:}" failed. No retries permitted until 2026-05-29 17:57:39.222542813 +0000 UTC m=+34.278336167 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9v5t" (UniqueName: "kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t") pod "network-check-target-cp2gr" (UID: "50d7a3bd-52e4-4d5d-8d79-fe64008680ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:23.464184 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:23.464091    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:23.464348 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:23.464227    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:24.463928 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:24.463891    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:24.464406 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:24.464012    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:25.464412 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:25.464382    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:25.464758 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:25.464468    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:26.463503 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.463481    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:26.463591 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:26.463575    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:26.630587 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.630391    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-28z92" event={"ID":"a054de51-319b-4ab1-b4ef-5f2d979ef630","Type":"ContainerStarted","Data":"822576c4e7a7a75dfc739b5105d87eadd961f31e37edea4bd720d0371b48666c"}
May 29 17:57:26.632578 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.632560    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 17:57:26.632867 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.632840    2575 generic.go:358] "Generic (PLEG): container finished" podID="f9ca71bf-3069-42a6-95d4-106519e08945" containerID="7dffaf020e40c5cc27b21cda1b574a24c5fd531b834dbac3c28ccd5d6eabbace" exitCode=1
May 29 17:57:26.632958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.632912    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" event={"ID":"f9ca71bf-3069-42a6-95d4-106519e08945","Type":"ContainerStarted","Data":"51d0689f2800d6141999a5df6c49555863d8a04322590af09c5a890dff4c3749"}
May 29 17:57:26.632958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.632933    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" event={"ID":"f9ca71bf-3069-42a6-95d4-106519e08945","Type":"ContainerStarted","Data":"30befe8185825b08bf004d6df4f2414b12e137898b9bf73b69a8d9448bb4d4f2"}
May 29 17:57:26.632958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.632945    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" event={"ID":"f9ca71bf-3069-42a6-95d4-106519e08945","Type":"ContainerStarted","Data":"d1b7fa8a3414d0b76e034c65e0ae44f55727694f1cd651c9fbc4cc8be03b02b3"}
May 29 17:57:26.632958 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.632956    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" event={"ID":"f9ca71bf-3069-42a6-95d4-106519e08945","Type":"ContainerDied","Data":"7dffaf020e40c5cc27b21cda1b574a24c5fd531b834dbac3c28ccd5d6eabbace"}
May 29 17:57:26.633132 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.632967    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" event={"ID":"f9ca71bf-3069-42a6-95d4-106519e08945","Type":"ContainerStarted","Data":"beb8a558566615b08f9a1854123e19d3f56af4281b032e6574198a72480c1d5d"}
May 29 17:57:26.634355 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.634335    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4spp5" event={"ID":"b4578642-65b0-478f-8f73-afc1f47c3009","Type":"ContainerStarted","Data":"319660d9ce2d0edda1ee2278a793bc39d256a2214ecacd716429b28e2065be37"}
May 29 17:57:26.635630 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.635608    2575 generic.go:358] "Generic (PLEG): container finished" podID="56fb789d-bd9e-4a88-958d-e8216ab60549" containerID="9f02b8b877c725ebe8b57aef5c3919c6a1aaf09606cbdc20a8bd3f6cd4935b2a" exitCode=0
May 29 17:57:26.635716 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.635667    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94hsf" event={"ID":"56fb789d-bd9e-4a88-958d-e8216ab60549","Type":"ContainerDied","Data":"9f02b8b877c725ebe8b57aef5c3919c6a1aaf09606cbdc20a8bd3f6cd4935b2a"}
May 29 17:57:26.637043 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.637019    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zctkf" event={"ID":"c21facb6-ed73-479a-93b2-19822a7c2620","Type":"ContainerStarted","Data":"a536c0743803d867bfbfc01efebb6c6aa299d7b8e72654c367a41ac8b6d9e1f3"}
May 29 17:57:26.638287 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.638264    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rq5ft" event={"ID":"2b3a52ac-9cde-4e35-b89c-f5ff5b50093c","Type":"ContainerStarted","Data":"ab5969ceef1abd4779d98197005283d4d0bec8818171bd1c778b6255a35eb2fd"}
May 29 17:57:26.639561 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.639532    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gcd7p" event={"ID":"0a29d4c2-23f1-4323-997a-1f5894d38144","Type":"ContainerStarted","Data":"322ad3df19c601e684688972c713b24c745c49ef9febe3ab49831e39915191db"}
May 29 17:57:26.641330 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.641306    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2" event={"ID":"9a355e8a-109a-4751-9421-a27eb4a47c78","Type":"ContainerStarted","Data":"52abf0341747a17c829caaa1569dbe8885fcd78e8ee71c1b83d7f5e803786a9c"}
May 29 17:57:26.659389 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.659351    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-167.ec2.internal" podStartSLOduration=20.659337686 podStartE2EDuration="20.659337686s" podCreationTimestamp="2026-05-29 17:57:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:57:09.619845394 +0000 UTC m=+4.675638760" watchObservedRunningTime="2026-05-29 17:57:26.659337686 +0000 UTC m=+21.715131050"
May 29 17:57:26.659943 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.659900    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-28z92" podStartSLOduration=3.595903395 podStartE2EDuration="21.65988703s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.886944337 +0000 UTC m=+2.942737676" lastFinishedPulling="2026-05-29 17:57:25.950927964 +0000 UTC m=+21.006721311" observedRunningTime="2026-05-29 17:57:26.659260322 +0000 UTC m=+21.715053711" watchObservedRunningTime="2026-05-29 17:57:26.65988703 +0000 UTC m=+21.715680393"
May 29 17:57:26.706952 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.706907    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-zctkf" podStartSLOduration=3.481687731 podStartE2EDuration="21.706893056s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.894894151 +0000 UTC m=+2.950687490" lastFinishedPulling="2026-05-29 17:57:26.120099471 +0000 UTC m=+21.175892815" observedRunningTime="2026-05-29 17:57:26.686478956 +0000 UTC m=+21.742272320" watchObservedRunningTime="2026-05-29 17:57:26.706893056 +0000 UTC m=+21.762686417"
May 29 17:57:26.725063 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.725005    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gcd7p" podStartSLOduration=7.574061555 podStartE2EDuration="21.72498761s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.891341334 +0000 UTC m=+2.947134675" lastFinishedPulling="2026-05-29 17:57:22.042267389 +0000 UTC m=+17.098060730" observedRunningTime="2026-05-29 17:57:26.706744858 +0000 UTC m=+21.762538221" watchObservedRunningTime="2026-05-29 17:57:26.72498761 +0000 UTC m=+21.780780973"
May 29 17:57:26.725411 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.725370    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-rq5ft" podStartSLOduration=3.667938082 podStartE2EDuration="21.725356631s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.891817551 +0000 UTC m=+2.947610891" lastFinishedPulling="2026-05-29 17:57:25.949236086 +0000 UTC m=+21.005029440" observedRunningTime="2026-05-29 17:57:26.724988163 +0000 UTC m=+21.780781526" watchObservedRunningTime="2026-05-29 17:57:26.725356631 +0000 UTC m=+21.781149994"
May 29 17:57:26.747131 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:26.747089    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4spp5" podStartSLOduration=7.599696154 podStartE2EDuration="21.74707571s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.894883741 +0000 UTC m=+2.950677081" lastFinishedPulling="2026-05-29 17:57:22.042263287 +0000 UTC m=+17.098056637" observedRunningTime="2026-05-29 17:57:26.746582898 +0000 UTC m=+21.802376262" watchObservedRunningTime="2026-05-29 17:57:26.74707571 +0000 UTC m=+21.802869073"
May 29 17:57:27.360610 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:27.360580    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 29 17:57:27.445583 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:27.445497    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-29T17:57:27.360606702Z","UUID":"aa44b58d-ff28-4ec3-ac2c-0c1047888456","Handler":null,"Name":"","Endpoint":""}
May 29 17:57:27.446810 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:27.446790    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 29 17:57:27.446810 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:27.446815    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 29 17:57:27.463668 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:27.463644    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:27.463787 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:27.463772    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:27.648756 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:27.648718    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2" event={"ID":"9a355e8a-109a-4751-9421-a27eb4a47c78","Type":"ContainerStarted","Data":"e2d8098c1738b037f95e0d00a82c976ec187e6c5e7fdbfd73ece1bc9a542a74c"}
May 29 17:57:27.650536 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:27.650503    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6jcqq" event={"ID":"2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb","Type":"ContainerStarted","Data":"26df3642b9eebb9191e69394369690797a6ea0f2ac90ff3c919e9273ce9a1732"}
May 29 17:57:27.653327 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:27.653307    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 17:57:27.653823 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:27.653741    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" event={"ID":"f9ca71bf-3069-42a6-95d4-106519e08945","Type":"ContainerStarted","Data":"f45f6b5907c05f605f7ef5f5f2157c0fd67be3902ff8d1829ca54038bf30763c"}
May 29 17:57:27.670031 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:27.669982    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6jcqq" podStartSLOduration=4.632269607 podStartE2EDuration="22.669968012s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.889637631 +0000 UTC m=+2.945430989" lastFinishedPulling="2026-05-29 17:57:25.92733604 +0000 UTC m=+20.983129394" observedRunningTime="2026-05-29 17:57:27.669690181 +0000 UTC m=+22.725483544" watchObservedRunningTime="2026-05-29 17:57:27.669968012 +0000 UTC m=+22.725761375"
May 29 17:57:28.464366 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:28.464182    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:28.464586 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:28.464451    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:28.656838 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:28.656801    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2" event={"ID":"9a355e8a-109a-4751-9421-a27eb4a47c78","Type":"ContainerStarted","Data":"65236b8ebbac3d3f3fe8ba16c1f2e74428d8a2dbd1c7cf806c208bc5cd766f37"}
May 29 17:57:28.678258 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:28.678209    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vjk2" podStartSLOduration=3.250871033 podStartE2EDuration="23.678193721s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.890411912 +0000 UTC m=+2.946205267" lastFinishedPulling="2026-05-29 17:57:28.317734615 +0000 UTC m=+23.373527955" observedRunningTime="2026-05-29 17:57:28.678046745 +0000 UTC m=+23.733840109" watchObservedRunningTime="2026-05-29 17:57:28.678193721 +0000 UTC m=+23.733987084"
May 29 17:57:29.463634 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:29.463598    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:29.463835 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:29.463735    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:29.491910 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:29.491884    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:29.492626 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:29.492582    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:29.662088 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:29.662062    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 17:57:29.662579 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:29.662555    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" event={"ID":"f9ca71bf-3069-42a6-95d4-106519e08945","Type":"ContainerStarted","Data":"c375dd485c87885a23d639e0e0df474980dc74233927a5fb08a875fbeb41d042"}
May 29 17:57:29.662991 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:29.662928    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:29.663258 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:29.663242    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gcd7p"
May 29 17:57:30.464248 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:30.464216    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:30.464427 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:30.464355    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:31.464337 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:31.464083    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:31.464870 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:31.464361    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:31.668112 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:31.668078    2575 generic.go:358] "Generic (PLEG): container finished" podID="56fb789d-bd9e-4a88-958d-e8216ab60549" containerID="a55c47ce48c6966e044be0b434da59b7e5bb2af590238ceb5064a37a6b02daf0" exitCode=0
May 29 17:57:31.668269 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:31.668183    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94hsf" event={"ID":"56fb789d-bd9e-4a88-958d-e8216ab60549","Type":"ContainerDied","Data":"a55c47ce48c6966e044be0b434da59b7e5bb2af590238ceb5064a37a6b02daf0"}
May 29 17:57:31.671486 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:31.671470    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 17:57:31.671817 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:31.671789    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" event={"ID":"f9ca71bf-3069-42a6-95d4-106519e08945","Type":"ContainerStarted","Data":"a5a6a2cd19fdf2dccc29716bdabdcb04f5610acc0890a97d2f25368a110e5cdd"}
May 29 17:57:31.672095 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:31.672068    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:31.672217 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:31.672107    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:31.672267 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:31.672249    2575 scope.go:117] "RemoveContainer" containerID="7dffaf020e40c5cc27b21cda1b574a24c5fd531b834dbac3c28ccd5d6eabbace"
May 29 17:57:31.688892 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:31.688872    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:32.465057 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:32.464439    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:32.465057 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:32.464566    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:32.677304 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:32.677230    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 17:57:32.677633 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:32.677605    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" event={"ID":"f9ca71bf-3069-42a6-95d4-106519e08945","Type":"ContainerStarted","Data":"472802a95d9dacd9fb26f5260b6e32e2dfea4905913a18b603e630de9b9b8efb"}
May 29 17:57:32.677816 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:32.677798    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:32.679677 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:32.679652    2575 generic.go:358] "Generic (PLEG): container finished" podID="56fb789d-bd9e-4a88-958d-e8216ab60549" containerID="62d3fe083b7271980bb5ab06509890fd1ceb29d0cb8e981b0459b4f6a4954334" exitCode=0
May 29 17:57:32.679785 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:32.679700    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94hsf" event={"ID":"56fb789d-bd9e-4a88-958d-e8216ab60549","Type":"ContainerDied","Data":"62d3fe083b7271980bb5ab06509890fd1ceb29d0cb8e981b0459b4f6a4954334"}
May 29 17:57:32.696176 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:32.696130    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:57:32.712784 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:32.712734    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9" podStartSLOduration=9.628409466 podStartE2EDuration="27.712719402s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.886560389 +0000 UTC m=+2.942353733" lastFinishedPulling="2026-05-29 17:57:25.970870327 +0000 UTC m=+21.026663669" observedRunningTime="2026-05-29 17:57:32.712171994 +0000 UTC m=+27.767965355" watchObservedRunningTime="2026-05-29 17:57:32.712719402 +0000 UTC m=+27.768512764"
May 29 17:57:33.185000 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:33.184968    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zsvdq"]
May 29 17:57:33.185185 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:33.185126    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:33.185280 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:33.185255    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:33.187711 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:33.187687    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cp2gr"]
May 29 17:57:33.187797 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:33.187779    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:33.187884 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:33.187863    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:33.683496 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:33.683454    2575 generic.go:358] "Generic (PLEG): container finished" podID="56fb789d-bd9e-4a88-958d-e8216ab60549" containerID="ed1409c8b82cd476b85491efff98e81a8eeb6136a51fb7063f35a1588133102d" exitCode=0
May 29 17:57:33.683864 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:33.683502    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94hsf" event={"ID":"56fb789d-bd9e-4a88-958d-e8216ab60549","Type":"ContainerDied","Data":"ed1409c8b82cd476b85491efff98e81a8eeb6136a51fb7063f35a1588133102d"}
May 29 17:57:34.464437 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:34.464404    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:34.464626 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:34.464523    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:34.464626 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:34.464579    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:34.464739 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:34.464676    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:36.463554 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:36.463523    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:36.464275 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:36.463523    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:36.464275 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:36.463639    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:36.464275 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:36.463743    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:38.463934 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.463898    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:38.464570 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.463907    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:38.464570 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:38.464020    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cp2gr" podUID="50d7a3bd-52e4-4d5d-8d79-fe64008680ab"
May 29 17:57:38.464570 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:38.464117    2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsvdq" podUID="19a8c87f-2157-4411-8e5a-b272ce1259f5"
May 29 17:57:38.803730 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.803643    2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-167.ec2.internal" event="NodeReady"
May 29 17:57:38.803884 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.803801    2575 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 29 17:57:38.850985 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.850945    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nl6fn"]
May 29 17:57:38.892299 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.892254    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c547r"]
May 29 17:57:38.892483 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.892381    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:38.895504 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.895474    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 29 17:57:38.895648 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.895590    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 29 17:57:38.895771 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.895747    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmxvm\""
May 29 17:57:38.906926 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.906889    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nl6fn"]
May 29 17:57:38.906926 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.906926    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c547r"]
May 29 17:57:38.907077 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.906990    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:38.910213 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.910193    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-85xq6\""
May 29 17:57:38.910332 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.910316    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 29 17:57:38.910430 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.910414    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 29 17:57:38.910490 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:38.910429    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 29 17:57:39.042728 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.042690    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9qd9\" (UniqueName: \"kubernetes.io/projected/eafe36c4-8553-4f04-be68-90e8803d6b0e-kube-api-access-z9qd9\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.042728 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.042731    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eafe36c4-8553-4f04-be68-90e8803d6b0e-config-volume\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.042999 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.042757    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eafe36c4-8553-4f04-be68-90e8803d6b0e-tmp-dir\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.042999 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.042779    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:39.042999 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.042799    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4nwk\" (UniqueName: \"kubernetes.io/projected/2c416805-3830-4517-ba34-2a7e9ec3eb8f-kube-api-access-m4nwk\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:39.042999 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.042847    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.143384 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.143355    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z9qd9\" (UniqueName: \"kubernetes.io/projected/eafe36c4-8553-4f04-be68-90e8803d6b0e-kube-api-access-z9qd9\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.143633 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.143388    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eafe36c4-8553-4f04-be68-90e8803d6b0e-config-volume\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.143633 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.143411    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eafe36c4-8553-4f04-be68-90e8803d6b0e-tmp-dir\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.143633 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.143428    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:39.143633 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.143452    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m4nwk\" (UniqueName: \"kubernetes.io/projected/2c416805-3830-4517-ba34-2a7e9ec3eb8f-kube-api-access-m4nwk\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:39.143633 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.143485    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:39.143633 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.143508    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.143633 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.143560    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:39.143633 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.143599    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:39.143633 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.143626    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs podName:19a8c87f-2157-4411-8e5a-b272ce1259f5 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:11.143606136 +0000 UTC m=+66.199399478 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs") pod "network-metrics-daemon-zsvdq" (UID: "19a8c87f-2157-4411-8e5a-b272ce1259f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 17:57:39.143993 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.143656    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:39.143993 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.143704    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls podName:eafe36c4-8553-4f04-be68-90e8803d6b0e nodeName:}" failed. No retries permitted until 2026-05-29 17:57:39.643687024 +0000 UTC m=+34.699480364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls") pod "dns-default-nl6fn" (UID: "eafe36c4-8553-4f04-be68-90e8803d6b0e") : secret "dns-default-metrics-tls" not found
May 29 17:57:39.143993 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.143726    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert podName:2c416805-3830-4517-ba34-2a7e9ec3eb8f nodeName:}" failed. No retries permitted until 2026-05-29 17:57:39.643713464 +0000 UTC m=+34.699506811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert") pod "ingress-canary-c547r" (UID: "2c416805-3830-4517-ba34-2a7e9ec3eb8f") : secret "canary-serving-cert" not found
May 29 17:57:39.143993 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.143770    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eafe36c4-8553-4f04-be68-90e8803d6b0e-tmp-dir\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.143993 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.143961    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eafe36c4-8553-4f04-be68-90e8803d6b0e-config-volume\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.155298 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.155275    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9qd9\" (UniqueName: \"kubernetes.io/projected/eafe36c4-8553-4f04-be68-90e8803d6b0e-kube-api-access-z9qd9\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.158699 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.158576    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4nwk\" (UniqueName: \"kubernetes.io/projected/2c416805-3830-4517-ba34-2a7e9ec3eb8f-kube-api-access-m4nwk\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:39.244670 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.244645    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j9v5t\" (UniqueName: \"kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t\") pod \"network-check-target-cp2gr\" (UID: \"50d7a3bd-52e4-4d5d-8d79-fe64008680ab\") " pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:39.244845 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.244788    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 17:57:39.244845 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.244806    2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 17:57:39.244845 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.244816    2575 projected.go:194] Error preparing data for projected volume kube-api-access-j9v5t for pod openshift-network-diagnostics/network-check-target-cp2gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:39.244950 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.244859    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t podName:50d7a3bd-52e4-4d5d-8d79-fe64008680ab nodeName:}" failed. No retries permitted until 2026-05-29 17:58:11.244846054 +0000 UTC m=+66.300639395 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9v5t" (UniqueName: "kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t") pod "network-check-target-cp2gr" (UID: "50d7a3bd-52e4-4d5d-8d79-fe64008680ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 17:57:39.647077 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.647051    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:39.647460 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.647097    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:39.647460 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.647226    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:39.647460 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.647230    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:39.647460 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.647277    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls podName:eafe36c4-8553-4f04-be68-90e8803d6b0e nodeName:}" failed. No retries permitted until 2026-05-29 17:57:40.647263481 +0000 UTC m=+35.703056821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls") pod "dns-default-nl6fn" (UID: "eafe36c4-8553-4f04-be68-90e8803d6b0e") : secret "dns-default-metrics-tls" not found
May 29 17:57:39.647460 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:39.647289    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert podName:2c416805-3830-4517-ba34-2a7e9ec3eb8f nodeName:}" failed. No retries permitted until 2026-05-29 17:57:40.647283364 +0000 UTC m=+35.703076704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert") pod "ingress-canary-c547r" (UID: "2c416805-3830-4517-ba34-2a7e9ec3eb8f") : secret "canary-serving-cert" not found
May 29 17:57:39.697626 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.697533    2575 generic.go:358] "Generic (PLEG): container finished" podID="56fb789d-bd9e-4a88-958d-e8216ab60549" containerID="6ecf31ccdb6b44171a66ca4671c9b0ae989592822a17e1427d342429b7c4d865" exitCode=0
May 29 17:57:39.697626 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:39.697592    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94hsf" event={"ID":"56fb789d-bd9e-4a88-958d-e8216ab60549","Type":"ContainerDied","Data":"6ecf31ccdb6b44171a66ca4671c9b0ae989592822a17e1427d342429b7c4d865"}
May 29 17:57:40.464508 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.464426    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:57:40.464650 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.464426    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:57:40.467464 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.467439    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 29 17:57:40.467687 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.467476    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 29 17:57:40.467687 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.467485    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-84j8s\""
May 29 17:57:40.467778 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.467771    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 29 17:57:40.468794 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.468780    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b7d95\""
May 29 17:57:40.652230 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.652201    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:40.652646 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.652255    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:40.652646 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:40.652352    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:40.652646 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:40.652400    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:40.652646 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:40.652427    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert podName:2c416805-3830-4517-ba34-2a7e9ec3eb8f nodeName:}" failed. No retries permitted until 2026-05-29 17:57:42.652410934 +0000 UTC m=+37.708204274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert") pod "ingress-canary-c547r" (UID: "2c416805-3830-4517-ba34-2a7e9ec3eb8f") : secret "canary-serving-cert" not found
May 29 17:57:40.652646 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:40.652490    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls podName:eafe36c4-8553-4f04-be68-90e8803d6b0e nodeName:}" failed. No retries permitted until 2026-05-29 17:57:42.652471097 +0000 UTC m=+37.708264447 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls") pod "dns-default-nl6fn" (UID: "eafe36c4-8553-4f04-be68-90e8803d6b0e") : secret "dns-default-metrics-tls" not found
May 29 17:57:40.703721 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.703696    2575 generic.go:358] "Generic (PLEG): container finished" podID="56fb789d-bd9e-4a88-958d-e8216ab60549" containerID="8bee03de4ff0db6821f9ffa5b137d0103bfa7fa0f2419ea18b5da89b42cfffd5" exitCode=0
May 29 17:57:40.703867 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:40.703728    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94hsf" event={"ID":"56fb789d-bd9e-4a88-958d-e8216ab60549","Type":"ContainerDied","Data":"8bee03de4ff0db6821f9ffa5b137d0103bfa7fa0f2419ea18b5da89b42cfffd5"}
May 29 17:57:41.708499 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:41.708469    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-94hsf" event={"ID":"56fb789d-bd9e-4a88-958d-e8216ab60549","Type":"ContainerStarted","Data":"b43c47f010f0b3c12713612cd77c3bdb4baeb0123ecbbdbb85d7675f071a3264"}
May 29 17:57:41.733712 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:41.733663    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-94hsf" podStartSLOduration=5.249772159 podStartE2EDuration="36.733649215s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:57:07.883999812 +0000 UTC m=+2.939793151" lastFinishedPulling="2026-05-29 17:57:39.367876867 +0000 UTC m=+34.423670207" observedRunningTime="2026-05-29 17:57:41.73244065 +0000 UTC m=+36.788234012" watchObservedRunningTime="2026-05-29 17:57:41.733649215 +0000 UTC m=+36.789442569"
May 29 17:57:42.667753 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:42.667713    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:42.667928 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:42.667770    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:42.667928 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:42.667854    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:42.667928 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:42.667911    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:42.667928 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:42.667919    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert podName:2c416805-3830-4517-ba34-2a7e9ec3eb8f nodeName:}" failed. No retries permitted until 2026-05-29 17:57:46.667903124 +0000 UTC m=+41.723696464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert") pod "ingress-canary-c547r" (UID: "2c416805-3830-4517-ba34-2a7e9ec3eb8f") : secret "canary-serving-cert" not found
May 29 17:57:42.668061 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:42.667950    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls podName:eafe36c4-8553-4f04-be68-90e8803d6b0e nodeName:}" failed. No retries permitted until 2026-05-29 17:57:46.66793956 +0000 UTC m=+41.723732900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls") pod "dns-default-nl6fn" (UID: "eafe36c4-8553-4f04-be68-90e8803d6b0e") : secret "dns-default-metrics-tls" not found
May 29 17:57:46.692788 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:46.692726    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:46.693261 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:46.692835    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:46.693261 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:46.692870    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:46.693261 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:46.692937    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls podName:eafe36c4-8553-4f04-be68-90e8803d6b0e nodeName:}" failed. No retries permitted until 2026-05-29 17:57:54.69292176 +0000 UTC m=+49.748715100 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls") pod "dns-default-nl6fn" (UID: "eafe36c4-8553-4f04-be68-90e8803d6b0e") : secret "dns-default-metrics-tls" not found
May 29 17:57:46.693261 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:46.692937    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:46.693261 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:46.692991    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert podName:2c416805-3830-4517-ba34-2a7e9ec3eb8f nodeName:}" failed. No retries permitted until 2026-05-29 17:57:54.692976157 +0000 UTC m=+49.748769511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert") pod "ingress-canary-c547r" (UID: "2c416805-3830-4517-ba34-2a7e9ec3eb8f") : secret "canary-serving-cert" not found
May 29 17:57:54.743138 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:54.743100    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:57:54.743716 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:57:54.743184    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:57:54.743716 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:54.743267    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:57:54.743716 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:54.743305    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:57:54.743716 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:54.743326    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert podName:2c416805-3830-4517-ba34-2a7e9ec3eb8f nodeName:}" failed. No retries permitted until 2026-05-29 17:58:10.743311878 +0000 UTC m=+65.799105217 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert") pod "ingress-canary-c547r" (UID: "2c416805-3830-4517-ba34-2a7e9ec3eb8f") : secret "canary-serving-cert" not found
May 29 17:57:54.743716 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:57:54.743356    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls podName:eafe36c4-8553-4f04-be68-90e8803d6b0e nodeName:}" failed. No retries permitted until 2026-05-29 17:58:10.74334119 +0000 UTC m=+65.799134545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls") pod "dns-default-nl6fn" (UID: "eafe36c4-8553-4f04-be68-90e8803d6b0e") : secret "dns-default-metrics-tls" not found
May 29 17:58:04.695298 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:04.695266    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cxqf9"
May 29 17:58:10.754575 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:10.754539    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:58:10.755020 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:10.754585    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:58:10.755020 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:10.754691    2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 17:58:10.755020 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:10.754714    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 17:58:10.755020 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:10.754755    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert podName:2c416805-3830-4517-ba34-2a7e9ec3eb8f nodeName:}" failed. No retries permitted until 2026-05-29 17:58:42.754739314 +0000 UTC m=+97.810532671 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert") pod "ingress-canary-c547r" (UID: "2c416805-3830-4517-ba34-2a7e9ec3eb8f") : secret "canary-serving-cert" not found
May 29 17:58:10.755020 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:10.754776    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls podName:eafe36c4-8553-4f04-be68-90e8803d6b0e nodeName:}" failed. No retries permitted until 2026-05-29 17:58:42.754762959 +0000 UTC m=+97.810556299 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls") pod "dns-default-nl6fn" (UID: "eafe36c4-8553-4f04-be68-90e8803d6b0e") : secret "dns-default-metrics-tls" not found
May 29 17:58:11.156851 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:11.156816    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:58:11.160126 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:11.160107    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 29 17:58:11.167692 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:11.167674    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 29 17:58:11.167761 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:11.167746    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs podName:19a8c87f-2157-4411-8e5a-b272ce1259f5 nodeName:}" failed. No retries permitted until 2026-05-29 17:59:15.167726216 +0000 UTC m=+130.223519556 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs") pod "network-metrics-daemon-zsvdq" (UID: "19a8c87f-2157-4411-8e5a-b272ce1259f5") : secret "metrics-daemon-secret" not found
May 29 17:58:11.257659 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:11.257628    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j9v5t\" (UniqueName: \"kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t\") pod \"network-check-target-cp2gr\" (UID: \"50d7a3bd-52e4-4d5d-8d79-fe64008680ab\") " pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:58:11.260645 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:11.260628    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 29 17:58:11.271188 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:11.271168    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 29 17:58:11.281936 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:11.281893    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9v5t\" (UniqueName: \"kubernetes.io/projected/50d7a3bd-52e4-4d5d-8d79-fe64008680ab-kube-api-access-j9v5t\") pod \"network-check-target-cp2gr\" (UID: \"50d7a3bd-52e4-4d5d-8d79-fe64008680ab\") " pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:58:11.381054 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:11.381024    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-84j8s\""
May 29 17:58:11.389396 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:11.389376    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:58:11.526733 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:11.526700    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cp2gr"]
May 29 17:58:11.530893 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:11.530850    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d7a3bd_52e4_4d5d_8d79_fe64008680ab.slice/crio-7666edaa0793a0dc08d10d2e81d3d12f362f26eb06e9273c1e897d3121e8ba68 WatchSource:0}: Error finding container 7666edaa0793a0dc08d10d2e81d3d12f362f26eb06e9273c1e897d3121e8ba68: Status 404 returned error can't find the container with id 7666edaa0793a0dc08d10d2e81d3d12f362f26eb06e9273c1e897d3121e8ba68
May 29 17:58:11.764689 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:11.764603    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cp2gr" event={"ID":"50d7a3bd-52e4-4d5d-8d79-fe64008680ab","Type":"ContainerStarted","Data":"7666edaa0793a0dc08d10d2e81d3d12f362f26eb06e9273c1e897d3121e8ba68"}
May 29 17:58:14.771804 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:14.771768    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cp2gr" event={"ID":"50d7a3bd-52e4-4d5d-8d79-fe64008680ab","Type":"ContainerStarted","Data":"dd69dd09fedea0942d9eb86e374e2d9f3de1baec77f8c110959c6e83a19d5fd9"}
May 29 17:58:14.772187 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:14.771924    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:58:14.788895 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:14.788852    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cp2gr" podStartSLOduration=67.271542344 podStartE2EDuration="1m9.788840695s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:58:11.532591419 +0000 UTC m=+66.588384759" lastFinishedPulling="2026-05-29 17:58:14.049889764 +0000 UTC m=+69.105683110" observedRunningTime="2026-05-29 17:58:14.788314982 +0000 UTC m=+69.844108345" watchObservedRunningTime="2026-05-29 17:58:14.788840695 +0000 UTC m=+69.844634092"
May 29 17:58:19.139521 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.139403    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"]
May 29 17:58:19.142013 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.141998    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.144501 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.144478    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
May 29 17:58:19.144623 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.144607    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
May 29 17:58:19.145793 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.145767    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
May 29 17:58:19.145793 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.145776    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
May 29 17:58:19.145923 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.145813    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hkxjp\""
May 29 17:58:19.153839 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.153820    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"]
May 29 17:58:19.156782 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.156766    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"]
May 29 17:58:19.159318 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.159294    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:19.161914 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.161890    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
May 29 17:58:19.162005 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.161969    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 29 17:58:19.162005 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.161994    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 29 17:58:19.162112 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.162081    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4ww4z\""
May 29 17:58:19.163511 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.163496    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
May 29 17:58:19.169690 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.169670    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"]
May 29 17:58:19.210760 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.210733    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5832977c-189f-4698-a7e6-790da79f5b5a-config\") pod \"service-ca-operator-69877444c-l7tvs\" (UID: \"5832977c-189f-4698-a7e6-790da79f5b5a\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.210866 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.210788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5832977c-189f-4698-a7e6-790da79f5b5a-serving-cert\") pod \"service-ca-operator-69877444c-l7tvs\" (UID: \"5832977c-189f-4698-a7e6-790da79f5b5a\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.210866 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.210845    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:19.210945 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.210877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:19.210945 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.210911    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqk9n\" (UniqueName: \"kubernetes.io/projected/5832977c-189f-4698-a7e6-790da79f5b5a-kube-api-access-cqk9n\") pod \"service-ca-operator-69877444c-l7tvs\" (UID: \"5832977c-189f-4698-a7e6-790da79f5b5a\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.210945 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.210939    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzgnj\" (UniqueName: \"kubernetes.io/projected/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-kube-api-access-wzgnj\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:19.259317 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.259291    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"]
May 29 17:58:19.261823 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.261808    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5b5cc84bff-6r2np"]
May 29 17:58:19.261963 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.261946    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.264552 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.264528    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
May 29 17:58:19.264552 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.264549    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
May 29 17:58:19.264692 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.264554    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.264692 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.264605    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
May 29 17:58:19.264922 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.264809    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-kzhmp\""
May 29 17:58:19.264922 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.264836    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
May 29 17:58:19.267768 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.267742    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
May 29 17:58:19.267871 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.267758    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
May 29 17:58:19.267927 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.267877    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
May 29 17:58:19.268013 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.267998    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-72d87\""
May 29 17:58:19.268089 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.268054    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
May 29 17:58:19.268306 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.268285    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 29 17:58:19.268418 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.268339    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 29 17:58:19.275406 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.275364    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"]
May 29 17:58:19.276171 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.276137    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5b5cc84bff-6r2np"]
May 29 17:58:19.311497 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311462    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5832977c-189f-4698-a7e6-790da79f5b5a-config\") pod \"service-ca-operator-69877444c-l7tvs\" (UID: \"5832977c-189f-4698-a7e6-790da79f5b5a\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.311598 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311507    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j42k2\" (UniqueName: \"kubernetes.io/projected/3aee17dc-28c0-425e-9e7b-694f66cd6459-kube-api-access-j42k2\") pod \"kube-storage-version-migrator-operator-96ddc787c-78pw7\" (UID: \"3aee17dc-28c0-425e-9e7b-694f66cd6459\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.311598 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-default-certificate\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.311598 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311545    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.311598 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311560    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvzc6\" (UniqueName: \"kubernetes.io/projected/c74f15b7-365c-4862-bbd3-ad53e42c1d36-kube-api-access-wvzc6\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.311598 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311589    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-stats-auth\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.311766 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311671    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5832977c-189f-4698-a7e6-790da79f5b5a-serving-cert\") pod \"service-ca-operator-69877444c-l7tvs\" (UID: \"5832977c-189f-4698-a7e6-790da79f5b5a\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.311766 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311702    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.311766 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311734    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:19.311860 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311784    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:19.311860 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3aee17dc-28c0-425e-9e7b-694f66cd6459-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-78pw7\" (UID: \"3aee17dc-28c0-425e-9e7b-694f66cd6459\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.311937 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311860    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cqk9n\" (UniqueName: \"kubernetes.io/projected/5832977c-189f-4698-a7e6-790da79f5b5a-kube-api-access-cqk9n\") pod \"service-ca-operator-69877444c-l7tvs\" (UID: \"5832977c-189f-4698-a7e6-790da79f5b5a\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.311937 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.311918    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 29 17:58:19.312030 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311964    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5832977c-189f-4698-a7e6-790da79f5b5a-config\") pod \"service-ca-operator-69877444c-l7tvs\" (UID: \"5832977c-189f-4698-a7e6-790da79f5b5a\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.312030 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.311974    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wzgnj\" (UniqueName: \"kubernetes.io/projected/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-kube-api-access-wzgnj\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:19.312030 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.312007    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls podName:049a52c2-66f9-47e2-b13b-bb28cc65d9b3 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:19.811987653 +0000 UTC m=+74.867781007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-757x8" (UID: "049a52c2-66f9-47e2-b13b-bb28cc65d9b3") : secret "cluster-monitoring-operator-tls" not found
May 29 17:58:19.312217 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.312072    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3aee17dc-28c0-425e-9e7b-694f66cd6459-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-78pw7\" (UID: \"3aee17dc-28c0-425e-9e7b-694f66cd6459\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.312376 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.312359    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:19.313844 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.313827    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5832977c-189f-4698-a7e6-790da79f5b5a-serving-cert\") pod \"service-ca-operator-69877444c-l7tvs\" (UID: \"5832977c-189f-4698-a7e6-790da79f5b5a\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.320842 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.320811    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqk9n\" (UniqueName: \"kubernetes.io/projected/5832977c-189f-4698-a7e6-790da79f5b5a-kube-api-access-cqk9n\") pod \"service-ca-operator-69877444c-l7tvs\" (UID: \"5832977c-189f-4698-a7e6-790da79f5b5a\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.321019 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.321001    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzgnj\" (UniqueName: \"kubernetes.io/projected/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-kube-api-access-wzgnj\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:19.365829 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.365802    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77c8d57b45-7zx6m"]
May 29 17:58:19.369490 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.369477    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.372168 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.372136    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 29 17:58:19.372275 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.372222    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h5q9d\""
May 29 17:58:19.372275 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.372222    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 29 17:58:19.372370 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.372340    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 29 17:58:19.381497 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.381447    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 29 17:58:19.397806 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.397744    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77c8d57b45-7zx6m"]
May 29 17:58:19.413102 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413079    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.413233 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413118    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.413233 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413141    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-bound-sa-token\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.413233 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413183    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-image-registry-private-configuration\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.413233 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-trusted-ca\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.413436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413263    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3aee17dc-28c0-425e-9e7b-694f66cd6459-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-78pw7\" (UID: \"3aee17dc-28c0-425e-9e7b-694f66cd6459\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.413436 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.413272    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:19.913251012 +0000 UTC m=+74.969044365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : configmap references non-existent config key: service-ca.crt
May 29 17:58:19.413436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413312    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aac1a9f-6772-4611-bc33-ac522d5de5d1-ca-trust-extracted\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.413436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413339    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-certificates\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.413436 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413375    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjbr4\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-kube-api-access-mjbr4\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.413678 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413466    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3aee17dc-28c0-425e-9e7b-694f66cd6459-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-78pw7\" (UID: \"3aee17dc-28c0-425e-9e7b-694f66cd6459\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.413678 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413501    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-installation-pull-secrets\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.413678 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413538    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j42k2\" (UniqueName: \"kubernetes.io/projected/3aee17dc-28c0-425e-9e7b-694f66cd6459-kube-api-access-j42k2\") pod \"kube-storage-version-migrator-operator-96ddc787c-78pw7\" (UID: \"3aee17dc-28c0-425e-9e7b-694f66cd6459\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.413678 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413653    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-default-certificate\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.413868 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413686    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.413868 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413714    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wvzc6\" (UniqueName: \"kubernetes.io/projected/c74f15b7-365c-4862-bbd3-ad53e42c1d36-kube-api-access-wvzc6\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.413868 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413760    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-stats-auth\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.413868 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.413842    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3aee17dc-28c0-425e-9e7b-694f66cd6459-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-78pw7\" (UID: \"3aee17dc-28c0-425e-9e7b-694f66cd6459\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.414063 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.414044    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 29 17:58:19.414114 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.414095    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:19.914079978 +0000 UTC m=+74.969873328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : secret "router-metrics-certs-default" not found
May 29 17:58:19.415781 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.415758    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3aee17dc-28c0-425e-9e7b-694f66cd6459-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-78pw7\" (UID: \"3aee17dc-28c0-425e-9e7b-694f66cd6459\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.416193 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.416177    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-stats-auth\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.416334 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.416313    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-default-certificate\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.425170 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.425130    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j42k2\" (UniqueName: \"kubernetes.io/projected/3aee17dc-28c0-425e-9e7b-694f66cd6459-kube-api-access-j42k2\") pod \"kube-storage-version-migrator-operator-96ddc787c-78pw7\" (UID: \"3aee17dc-28c0-425e-9e7b-694f66cd6459\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.425371 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.425355    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvzc6\" (UniqueName: \"kubernetes.io/projected/c74f15b7-365c-4862-bbd3-ad53e42c1d36-kube-api-access-wvzc6\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.450723 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.450705    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"
May 29 17:58:19.514896 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.514854    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-image-registry-private-configuration\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.514896 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.514893    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-trusted-ca\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.515127 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.514927    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aac1a9f-6772-4611-bc33-ac522d5de5d1-ca-trust-extracted\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.515127 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.514949    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-certificates\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.515127 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.514968    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mjbr4\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-kube-api-access-mjbr4\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.515315 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.515148    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-installation-pull-secrets\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.515315 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.515305    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.515417 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.515346    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-bound-sa-token\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.515467 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.515427    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 29 17:58:19.515467 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.515442    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c8d57b45-7zx6m: secret "image-registry-tls" not found
May 29 17:58:19.515563 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.515505    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls podName:4aac1a9f-6772-4611-bc33-ac522d5de5d1 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:20.015485696 +0000 UTC m=+75.071279053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls") pod "image-registry-77c8d57b45-7zx6m" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1") : secret "image-registry-tls" not found
May 29 17:58:19.516565 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.515919    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aac1a9f-6772-4611-bc33-ac522d5de5d1-ca-trust-extracted\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.516565 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.515992    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-certificates\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.516565 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.516511    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-trusted-ca\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.516987 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.516956    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-image-registry-private-configuration\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.517231 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.517208    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-installation-pull-secrets\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.524101 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.524075    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-bound-sa-token\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.525088 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.525067    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjbr4\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-kube-api-access-mjbr4\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:19.565982 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.565952    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs"]
May 29 17:58:19.568638 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:19.568615    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5832977c_189f_4698_a7e6_790da79f5b5a.slice/crio-aeaec9a2a61cb505238557616af74df4a9d99b205a34d6c1a7ac92ad62142256 WatchSource:0}: Error finding container aeaec9a2a61cb505238557616af74df4a9d99b205a34d6c1a7ac92ad62142256: Status 404 returned error can't find the container with id aeaec9a2a61cb505238557616af74df4a9d99b205a34d6c1a7ac92ad62142256
May 29 17:58:19.571660 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.571642    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"
May 29 17:58:19.684678 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.684468    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7"]
May 29 17:58:19.686856 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:19.686832    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aee17dc_28c0_425e_9e7b_694f66cd6459.slice/crio-21f7bd1dc3cf7de8852ce7ca0b260dc146d242eeefbe11ba7c65804bead1d934 WatchSource:0}: Error finding container 21f7bd1dc3cf7de8852ce7ca0b260dc146d242eeefbe11ba7c65804bead1d934: Status 404 returned error can't find the container with id 21f7bd1dc3cf7de8852ce7ca0b260dc146d242eeefbe11ba7c65804bead1d934
May 29 17:58:19.782643 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.782613    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7" event={"ID":"3aee17dc-28c0-425e-9e7b-694f66cd6459","Type":"ContainerStarted","Data":"21f7bd1dc3cf7de8852ce7ca0b260dc146d242eeefbe11ba7c65804bead1d934"}
May 29 17:58:19.783576 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.783552    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs" event={"ID":"5832977c-189f-4698-a7e6-790da79f5b5a","Type":"ContainerStarted","Data":"aeaec9a2a61cb505238557616af74df4a9d99b205a34d6c1a7ac92ad62142256"}
May 29 17:58:19.818329 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.818302    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:19.818429 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.818397    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 29 17:58:19.818475 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.818443    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls podName:049a52c2-66f9-47e2-b13b-bb28cc65d9b3 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:20.818429977 +0000 UTC m=+75.874223316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-757x8" (UID: "049a52c2-66f9-47e2-b13b-bb28cc65d9b3") : secret "cluster-monitoring-operator-tls" not found
May 29 17:58:19.919608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.919563    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.919751 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:19.919710    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:19.919797 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.919711    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 29 17:58:19.919847 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.919837    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:20.91982012 +0000 UTC m=+75.975613463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : secret "router-metrics-certs-default" not found
May 29 17:58:19.919894 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:19.919853    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:20.91984684 +0000 UTC m=+75.975640180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : configmap references non-existent config key: service-ca.crt
May 29 17:58:20.020766 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:20.020682    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:20.020909 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:20.020826    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 29 17:58:20.020909 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:20.020844    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c8d57b45-7zx6m: secret "image-registry-tls" not found
May 29 17:58:20.020909 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:20.020892    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls podName:4aac1a9f-6772-4611-bc33-ac522d5de5d1 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:21.020878184 +0000 UTC m=+76.076671525 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls") pod "image-registry-77c8d57b45-7zx6m" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1") : secret "image-registry-tls" not found
May 29 17:58:20.827437 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:20.827403    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:20.827861 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:20.827550    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 29 17:58:20.827861 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:20.827616    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls podName:049a52c2-66f9-47e2-b13b-bb28cc65d9b3 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:22.827602122 +0000 UTC m=+77.883395465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-757x8" (UID: "049a52c2-66f9-47e2-b13b-bb28cc65d9b3") : secret "cluster-monitoring-operator-tls" not found
May 29 17:58:20.927747 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:20.927708    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:20.927901 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:20.927770    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:20.927901 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:20.927860    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 29 17:58:20.927966 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:20.927924    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:22.927903918 +0000 UTC m=+77.983697258 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : configmap references non-existent config key: service-ca.crt
May 29 17:58:20.927966 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:20.927942    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:22.927935399 +0000 UTC m=+77.983728738 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : secret "router-metrics-certs-default" not found
May 29 17:58:21.029209 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:21.029167    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:21.029379 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:21.029301    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 29 17:58:21.029379 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:21.029320    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c8d57b45-7zx6m: secret "image-registry-tls" not found
May 29 17:58:21.029379 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:21.029379    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls podName:4aac1a9f-6772-4611-bc33-ac522d5de5d1 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:23.029364887 +0000 UTC m=+78.085158227 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls") pod "image-registry-77c8d57b45-7zx6m" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1") : secret "image-registry-tls" not found
May 29 17:58:22.790608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:22.790572    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7" event={"ID":"3aee17dc-28c0-425e-9e7b-694f66cd6459","Type":"ContainerStarted","Data":"c05153fd31eef01c882b9dc90ea0ba765b87eb37c93ea929b3ed808eb227c63d"}
May 29 17:58:22.791874 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:22.791849    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs" event={"ID":"5832977c-189f-4698-a7e6-790da79f5b5a","Type":"ContainerStarted","Data":"9fbf5298c025cf8d4d04ace19ba620f3f4e2ad5e768e8df629f51d7a02b6a8dd"}
May 29 17:58:22.819881 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:22.819835    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7" podStartSLOduration=1.2814200869999999 podStartE2EDuration="3.819820477s" podCreationTimestamp="2026-05-29 17:58:19 +0000 UTC" firstStartedPulling="2026-05-29 17:58:19.68879176 +0000 UTC m=+74.744585100" lastFinishedPulling="2026-05-29 17:58:22.227192147 +0000 UTC m=+77.282985490" observedRunningTime="2026-05-29 17:58:22.819275964 +0000 UTC m=+77.875069327" watchObservedRunningTime="2026-05-29 17:58:22.819820477 +0000 UTC m=+77.875613829"
May 29 17:58:22.846244 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:22.846211    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:22.846397 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:22.846378    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 29 17:58:22.846475 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:22.846440    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls podName:049a52c2-66f9-47e2-b13b-bb28cc65d9b3 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:26.846418714 +0000 UTC m=+81.902212055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-757x8" (UID: "049a52c2-66f9-47e2-b13b-bb28cc65d9b3") : secret "cluster-monitoring-operator-tls" not found
May 29 17:58:22.947493 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:22.947455    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:22.947680 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:22.947551    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:22.947680 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:22.947614    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 29 17:58:22.947775 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:22.947687    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:26.947666495 +0000 UTC m=+82.003459851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : configmap references non-existent config key: service-ca.crt
May 29 17:58:22.947775 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:22.947719    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:26.947706978 +0000 UTC m=+82.003500331 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : secret "router-metrics-certs-default" not found
May 29 17:58:23.048813 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:23.048719    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:23.048948 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:23.048878    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 29 17:58:23.048948 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:23.048901    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c8d57b45-7zx6m: secret "image-registry-tls" not found
May 29 17:58:23.049044 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:23.048969    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls podName:4aac1a9f-6772-4611-bc33-ac522d5de5d1 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:27.048949572 +0000 UTC m=+82.104742915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls") pod "image-registry-77c8d57b45-7zx6m" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1") : secret "image-registry-tls" not found
May 29 17:58:25.501406 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:25.501374    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rq5ft_2b3a52ac-9cde-4e35-b89c-f5ff5b50093c/dns-node-resolver/0.log"
May 29 17:58:26.101113 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:26.101085    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4spp5_b4578642-65b0-478f-8f73-afc1f47c3009/node-ca/0.log"
May 29 17:58:26.880029 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:26.879992    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:26.880434 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:26.880129    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 29 17:58:26.880434 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:26.880213    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls podName:049a52c2-66f9-47e2-b13b-bb28cc65d9b3 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:34.880197076 +0000 UTC m=+89.935990419 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-757x8" (UID: "049a52c2-66f9-47e2-b13b-bb28cc65d9b3") : secret "cluster-monitoring-operator-tls" not found
May 29 17:58:26.980782 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:26.980744    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:26.980930 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:26.980815    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:26.980930 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:26.980888    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 29 17:58:26.981002 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:26.980948    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:34.980934448 +0000 UTC m=+90.036727792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : secret "router-metrics-certs-default" not found
May 29 17:58:26.981002 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:26.980961    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:34.980955556 +0000 UTC m=+90.036748896 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : configmap references non-existent config key: service-ca.crt
May 29 17:58:27.081494 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:27.081452    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:27.081661 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:27.081583    2575 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 29 17:58:27.081661 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:27.081601    2575 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77c8d57b45-7zx6m: secret "image-registry-tls" not found
May 29 17:58:27.081661 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:27.081650    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls podName:4aac1a9f-6772-4611-bc33-ac522d5de5d1 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:35.081637172 +0000 UTC m=+90.137430511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls") pod "image-registry-77c8d57b45-7zx6m" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1") : secret "image-registry-tls" not found
May 29 17:58:34.944014 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:34.943971    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:34.944438 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:34.944087    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 29 17:58:34.944438 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:34.944150    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls podName:049a52c2-66f9-47e2-b13b-bb28cc65d9b3 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:50.944135875 +0000 UTC m=+105.999929216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-757x8" (UID: "049a52c2-66f9-47e2-b13b-bb28cc65d9b3") : secret "cluster-monitoring-operator-tls" not found
May 29 17:58:35.044528 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.044486    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:35.044662 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.044571    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:35.044714 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:35.044700    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle podName:c74f15b7-365c-4862-bbd3-ad53e42c1d36 nodeName:}" failed. No retries permitted until 2026-05-29 17:58:51.044680734 +0000 UTC m=+106.100474095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle") pod "router-default-5b5cc84bff-6r2np" (UID: "c74f15b7-365c-4862-bbd3-ad53e42c1d36") : configmap references non-existent config key: service-ca.crt
May 29 17:58:35.046993 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.046963    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c74f15b7-365c-4862-bbd3-ad53e42c1d36-metrics-certs\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:35.145614 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.145561    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:35.147832 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.147804    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls\") pod \"image-registry-77c8d57b45-7zx6m\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") " pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:35.279507 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.279423    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:35.397384 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.397334    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs" podStartSLOduration=13.727967097 podStartE2EDuration="16.397318638s" podCreationTimestamp="2026-05-29 17:58:19 +0000 UTC" firstStartedPulling="2026-05-29 17:58:19.570323796 +0000 UTC m=+74.626117135" lastFinishedPulling="2026-05-29 17:58:22.239675336 +0000 UTC m=+77.295468676" observedRunningTime="2026-05-29 17:58:22.850972613 +0000 UTC m=+77.906765976" watchObservedRunningTime="2026-05-29 17:58:35.397318638 +0000 UTC m=+90.453111999"
May 29 17:58:35.398102 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.398084    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77c8d57b45-7zx6m"]
May 29 17:58:35.400764 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:35.400734    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aac1a9f_6772_4611_bc33_ac522d5de5d1.slice/crio-2e30430b76abbd665a3620d5b8d877827de0bf424881077ab6ab7aa628b90d9a WatchSource:0}: Error finding container 2e30430b76abbd665a3620d5b8d877827de0bf424881077ab6ab7aa628b90d9a: Status 404 returned error can't find the container with id 2e30430b76abbd665a3620d5b8d877827de0bf424881077ab6ab7aa628b90d9a
May 29 17:58:35.818565 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.818520    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m" event={"ID":"4aac1a9f-6772-4611-bc33-ac522d5de5d1","Type":"ContainerStarted","Data":"7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333"}
May 29 17:58:35.818565 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.818563    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m" event={"ID":"4aac1a9f-6772-4611-bc33-ac522d5de5d1","Type":"ContainerStarted","Data":"2e30430b76abbd665a3620d5b8d877827de0bf424881077ab6ab7aa628b90d9a"}
May 29 17:58:35.818804 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.818665    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:35.837933 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:35.837890    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m" podStartSLOduration=16.837877076 podStartE2EDuration="16.837877076s" podCreationTimestamp="2026-05-29 17:58:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:58:35.837359154 +0000 UTC m=+90.893152516" watchObservedRunningTime="2026-05-29 17:58:35.837877076 +0000 UTC m=+90.893670438"
May 29 17:58:42.805141 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:42.805090    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:58:42.805514 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:42.805212    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:58:42.807418 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:42.807395    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eafe36c4-8553-4f04-be68-90e8803d6b0e-metrics-tls\") pod \"dns-default-nl6fn\" (UID: \"eafe36c4-8553-4f04-be68-90e8803d6b0e\") " pod="openshift-dns/dns-default-nl6fn"
May 29 17:58:42.807526 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:42.807478    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c416805-3830-4517-ba34-2a7e9ec3eb8f-cert\") pod \"ingress-canary-c547r\" (UID: \"2c416805-3830-4517-ba34-2a7e9ec3eb8f\") " pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:58:42.820624 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:42.820603    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-85xq6\""
May 29 17:58:42.828164 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:42.828135    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c547r"
May 29 17:58:42.941257 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:42.941229    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c547r"]
May 29 17:58:42.944220 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:42.944191    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c416805_3830_4517_ba34_2a7e9ec3eb8f.slice/crio-c5b4f57995c8325d8a6789532c5a2cdbeb6fd67ebbfaaab44c8ca21c496f3329 WatchSource:0}: Error finding container c5b4f57995c8325d8a6789532c5a2cdbeb6fd67ebbfaaab44c8ca21c496f3329: Status 404 returned error can't find the container with id c5b4f57995c8325d8a6789532c5a2cdbeb6fd67ebbfaaab44c8ca21c496f3329
May 29 17:58:43.105171 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:43.105078    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmxvm\""
May 29 17:58:43.113387 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:43.113369    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nl6fn"
May 29 17:58:43.230786 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:43.230756    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nl6fn"]
May 29 17:58:43.234447 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:43.234420    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeafe36c4_8553_4f04_be68_90e8803d6b0e.slice/crio-78967450d897c36baadb0d8c1b971ac2419c0832375d49769eeadf98a6428c19 WatchSource:0}: Error finding container 78967450d897c36baadb0d8c1b971ac2419c0832375d49769eeadf98a6428c19: Status 404 returned error can't find the container with id 78967450d897c36baadb0d8c1b971ac2419c0832375d49769eeadf98a6428c19
May 29 17:58:43.834941 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:43.834900    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nl6fn" event={"ID":"eafe36c4-8553-4f04-be68-90e8803d6b0e","Type":"ContainerStarted","Data":"78967450d897c36baadb0d8c1b971ac2419c0832375d49769eeadf98a6428c19"}
May 29 17:58:43.836325 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:43.836293    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c547r" event={"ID":"2c416805-3830-4517-ba34-2a7e9ec3eb8f","Type":"ContainerStarted","Data":"c5b4f57995c8325d8a6789532c5a2cdbeb6fd67ebbfaaab44c8ca21c496f3329"}
May 29 17:58:45.776939 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:45.776910    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cp2gr"
May 29 17:58:45.841877 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:45.841839    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nl6fn" event={"ID":"eafe36c4-8553-4f04-be68-90e8803d6b0e","Type":"ContainerStarted","Data":"cedef7c77fdea530b841f3daf51797b49be6a6e0308b4ee264e3d9efd74f22da"}
May 29 17:58:45.841877 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:45.841877    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nl6fn" event={"ID":"eafe36c4-8553-4f04-be68-90e8803d6b0e","Type":"ContainerStarted","Data":"7fb77b74f861f140a7c39c6bf979b93735205a4cdda037a096b491724838244b"}
May 29 17:58:45.842251 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:45.841955    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nl6fn"
May 29 17:58:45.843100 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:45.843081    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c547r" event={"ID":"2c416805-3830-4517-ba34-2a7e9ec3eb8f","Type":"ContainerStarted","Data":"527799db2630c40e59fd0cedac14550f490c133bc937b7c5de8fe6880fc93d23"}
May 29 17:58:45.859465 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:45.859424    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-nl6fn" podStartSLOduration=66.230729215 podStartE2EDuration="1m7.859410573s" podCreationTimestamp="2026-05-29 17:57:38 +0000 UTC" firstStartedPulling="2026-05-29 17:58:43.236126918 +0000 UTC m=+98.291920258" lastFinishedPulling="2026-05-29 17:58:44.864808277 +0000 UTC m=+99.920601616" observedRunningTime="2026-05-29 17:58:45.858873319 +0000 UTC m=+100.914666674" watchObservedRunningTime="2026-05-29 17:58:45.859410573 +0000 UTC m=+100.915203936"
May 29 17:58:45.874422 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:45.874375    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c547r" podStartSLOduration=65.957008704 podStartE2EDuration="1m7.874358919s" podCreationTimestamp="2026-05-29 17:57:38 +0000 UTC" firstStartedPulling="2026-05-29 17:58:42.945883395 +0000 UTC m=+98.001676738" lastFinishedPulling="2026-05-29 17:58:44.863233613 +0000 UTC m=+99.919026953" observedRunningTime="2026-05-29 17:58:45.873799469 +0000 UTC m=+100.929592831" watchObservedRunningTime="2026-05-29 17:58:45.874358919 +0000 UTC m=+100.930152284"
May 29 17:58:48.606507 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.606473    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77c8d57b45-7zx6m"]
May 29 17:58:48.627911 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.627884    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-khp6q"]
May 29 17:58:48.630574 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.630559    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.636419 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.636311    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 29 17:58:48.636578 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.636559    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 29 17:58:48.637134 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.636635    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 29 17:58:48.637245 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.636741    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 29 17:58:48.637245 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.636874    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6xcjf\""
May 29 17:58:48.643741 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.643717    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-khp6q"]
May 29 17:58:48.753794 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.753766    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.753927 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.753798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.753927 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.753836    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-data-volume\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.753927 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.753917    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-crio-socket\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.754048 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.753944    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j6xf\" (UniqueName: \"kubernetes.io/projected/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-kube-api-access-7j6xf\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.855147 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.855121    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-crio-socket\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.855319 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.855176    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7j6xf\" (UniqueName: \"kubernetes.io/projected/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-kube-api-access-7j6xf\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.855319 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.855210    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.855319 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.855230    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.855319 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.855272    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-data-volume\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.855319 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.855228    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-crio-socket\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.855679 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.855658    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-data-volume\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.855847 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.855832    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.857461 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.857404    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.865937 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.865919    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j6xf\" (UniqueName: \"kubernetes.io/projected/2f5bed7a-be5a-4b55-aa97-e52ac73285d2-kube-api-access-7j6xf\") pod \"insights-runtime-extractor-khp6q\" (UID: \"2f5bed7a-be5a-4b55-aa97-e52ac73285d2\") " pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:48.939602 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:48.939576    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-khp6q"
May 29 17:58:49.059233 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:49.059202    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-khp6q"]
May 29 17:58:49.063276 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:49.063252    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5bed7a_be5a_4b55_aa97_e52ac73285d2.slice/crio-322f6d679cee0ffd593228cc9e33d136eb8878128841f9c8dde4b203b7c22133 WatchSource:0}: Error finding container 322f6d679cee0ffd593228cc9e33d136eb8878128841f9c8dde4b203b7c22133: Status 404 returned error can't find the container with id 322f6d679cee0ffd593228cc9e33d136eb8878128841f9c8dde4b203b7c22133
May 29 17:58:49.854708 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:49.854677    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-khp6q" event={"ID":"2f5bed7a-be5a-4b55-aa97-e52ac73285d2","Type":"ContainerStarted","Data":"92f61e6f3e451b01d68e93f45a28a8aee5ea0a741ee241d11632847d566fc55b"}
May 29 17:58:49.854708 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:49.854711    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-khp6q" event={"ID":"2f5bed7a-be5a-4b55-aa97-e52ac73285d2","Type":"ContainerStarted","Data":"322f6d679cee0ffd593228cc9e33d136eb8878128841f9c8dde4b203b7c22133"}
May 29 17:58:50.858828 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:50.858788    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-khp6q" event={"ID":"2f5bed7a-be5a-4b55-aa97-e52ac73285d2","Type":"ContainerStarted","Data":"6014dac60535a363c0babb20e207f4e8ea330b62d9ab64e3b0b356c125b08d52"}
May 29 17:58:50.971198 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:50.971140    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:50.973887 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:50.973862    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/049a52c2-66f9-47e2-b13b-bb28cc65d9b3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-757x8\" (UID: \"049a52c2-66f9-47e2-b13b-bb28cc65d9b3\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:51.072251 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.072211    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:51.072923 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.072901    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c74f15b7-365c-4862-bbd3-ad53e42c1d36-service-ca-bundle\") pod \"router-default-5b5cc84bff-6r2np\" (UID: \"c74f15b7-365c-4862-bbd3-ad53e42c1d36\") " pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:51.077216 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.077191    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:51.267320 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.267279    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"
May 29 17:58:51.456777 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.456748    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8"]
May 29 17:58:51.461103 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:51.461079    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049a52c2_66f9_47e2_b13b_bb28cc65d9b3.slice/crio-a5c2b4323450124a67b7ec4879c46e00138730bb473182ccd95af7d3ffa3bd3f WatchSource:0}: Error finding container a5c2b4323450124a67b7ec4879c46e00138730bb473182ccd95af7d3ffa3bd3f: Status 404 returned error can't find the container with id a5c2b4323450124a67b7ec4879c46e00138730bb473182ccd95af7d3ffa3bd3f
May 29 17:58:51.476604 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.476577    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5b5cc84bff-6r2np"]
May 29 17:58:51.480451 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:51.480429    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc74f15b7_365c_4862_bbd3_ad53e42c1d36.slice/crio-5e191630066df05f240d124078231001766aebe51e881128790396a017fe324a WatchSource:0}: Error finding container 5e191630066df05f240d124078231001766aebe51e881128790396a017fe324a: Status 404 returned error can't find the container with id 5e191630066df05f240d124078231001766aebe51e881128790396a017fe324a
May 29 17:58:51.862748 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.862654    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-khp6q" event={"ID":"2f5bed7a-be5a-4b55-aa97-e52ac73285d2","Type":"ContainerStarted","Data":"d6c262d01683fe12fcb962a0d194161ddd7af0411fa0191457dc366e6e9197ab"}
May 29 17:58:51.863980 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.863955    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5b5cc84bff-6r2np" event={"ID":"c74f15b7-365c-4862-bbd3-ad53e42c1d36","Type":"ContainerStarted","Data":"607cbd2660c79711e093ece5352a77eef4d5352a9ddb4206fafd14110ac58163"}
May 29 17:58:51.864091 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.863986    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5b5cc84bff-6r2np" event={"ID":"c74f15b7-365c-4862-bbd3-ad53e42c1d36","Type":"ContainerStarted","Data":"5e191630066df05f240d124078231001766aebe51e881128790396a017fe324a"}
May 29 17:58:51.864872 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.864851    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8" event={"ID":"049a52c2-66f9-47e2-b13b-bb28cc65d9b3","Type":"ContainerStarted","Data":"a5c2b4323450124a67b7ec4879c46e00138730bb473182ccd95af7d3ffa3bd3f"}
May 29 17:58:51.879705 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.879664    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-khp6q" podStartSLOduration=1.600232282 podStartE2EDuration="3.879651862s" podCreationTimestamp="2026-05-29 17:58:48 +0000 UTC" firstStartedPulling="2026-05-29 17:58:49.108437852 +0000 UTC m=+104.164231195" lastFinishedPulling="2026-05-29 17:58:51.387857429 +0000 UTC m=+106.443650775" observedRunningTime="2026-05-29 17:58:51.878667351 +0000 UTC m=+106.934460725" watchObservedRunningTime="2026-05-29 17:58:51.879651862 +0000 UTC m=+106.935445217"
May 29 17:58:51.896610 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:51.896573    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-5b5cc84bff-6r2np" podStartSLOduration=32.896563309 podStartE2EDuration="32.896563309s" podCreationTimestamp="2026-05-29 17:58:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 17:58:51.896025311 +0000 UTC m=+106.951818697" watchObservedRunningTime="2026-05-29 17:58:51.896563309 +0000 UTC m=+106.952356671"
May 29 17:58:52.077424 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:52.077382    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:52.079876 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:52.079854    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:52.868222 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:52.868184    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:52.869234 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:52.869213    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5b5cc84bff-6r2np"
May 29 17:58:53.710480 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:53.710444    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx"]
May 29 17:58:53.714481 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:53.714460    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx"
May 29 17:58:53.717345 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:53.717327    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
May 29 17:58:53.718492 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:53.718474    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vsm98\""
May 29 17:58:53.724123 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:53.724103    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx"]
May 29 17:58:53.875933 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:53.875900    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8" event={"ID":"049a52c2-66f9-47e2-b13b-bb28cc65d9b3","Type":"ContainerStarted","Data":"050bce5bc0ee4513f7744bce232958d11a92874efa9f7d5c4790d2410abab85b"}
May 29 17:58:53.893596 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:53.893565    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7a2cc0b6-442f-4e15-9f9f-1b82c6cacf8f-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-v8flx\" (UID: \"7a2cc0b6-442f-4e15-9f9f-1b82c6cacf8f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx"
May 29 17:58:53.994452 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:53.994375    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7a2cc0b6-442f-4e15-9f9f-1b82c6cacf8f-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-v8flx\" (UID: \"7a2cc0b6-442f-4e15-9f9f-1b82c6cacf8f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx"
May 29 17:58:53.996987 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:53.996960    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7a2cc0b6-442f-4e15-9f9f-1b82c6cacf8f-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-v8flx\" (UID: \"7a2cc0b6-442f-4e15-9f9f-1b82c6cacf8f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx"
May 29 17:58:54.023363 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:54.023340    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx"
May 29 17:58:54.145585 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:54.145534    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-757x8" podStartSLOduration=33.384740391 podStartE2EDuration="35.145512167s" podCreationTimestamp="2026-05-29 17:58:19 +0000 UTC" firstStartedPulling="2026-05-29 17:58:51.463102512 +0000 UTC m=+106.518895853" lastFinishedPulling="2026-05-29 17:58:53.223874289 +0000 UTC m=+108.279667629" observedRunningTime="2026-05-29 17:58:53.893774657 +0000 UTC m=+108.949568046" watchObservedRunningTime="2026-05-29 17:58:54.145512167 +0000 UTC m=+109.201305532"
May 29 17:58:54.145826 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:54.145812    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx"]
May 29 17:58:54.148954 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:54.148928    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2cc0b6_442f_4e15_9f9f_1b82c6cacf8f.slice/crio-29e1800bb6bfc1b7fe01172594ec76d3d54136fa47ce44461cfa9978ae9a603a WatchSource:0}: Error finding container 29e1800bb6bfc1b7fe01172594ec76d3d54136fa47ce44461cfa9978ae9a603a: Status 404 returned error can't find the container with id 29e1800bb6bfc1b7fe01172594ec76d3d54136fa47ce44461cfa9978ae9a603a
May 29 17:58:54.880042 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:54.880004    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx" event={"ID":"7a2cc0b6-442f-4e15-9f9f-1b82c6cacf8f","Type":"ContainerStarted","Data":"29e1800bb6bfc1b7fe01172594ec76d3d54136fa47ce44461cfa9978ae9a603a"}
May 29 17:58:55.847060 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:55.847032    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nl6fn"
May 29 17:58:55.886286 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:55.884952    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx" event={"ID":"7a2cc0b6-442f-4e15-9f9f-1b82c6cacf8f","Type":"ContainerStarted","Data":"29cfb00540c312763cf4f17d42cb26fcd885df8aefb89f8cf5a9f86d8b9c8125"}
May 29 17:58:55.886811 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:55.886770    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx"
May 29 17:58:55.904566 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:55.904286    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx"
May 29 17:58:55.906600 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:55.906552    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-v8flx" podStartSLOduration=1.800228772 podStartE2EDuration="2.90653725s" podCreationTimestamp="2026-05-29 17:58:53 +0000 UTC" firstStartedPulling="2026-05-29 17:58:54.150908055 +0000 UTC m=+109.206701395" lastFinishedPulling="2026-05-29 17:58:55.25721653 +0000 UTC m=+110.313009873" observedRunningTime="2026-05-29 17:58:55.905570715 +0000 UTC m=+110.961364078" watchObservedRunningTime="2026-05-29 17:58:55.90653725 +0000 UTC m=+110.962330612"
May 29 17:58:56.767398 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.767364    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"]
May 29 17:58:56.770442 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.770420    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:56.773165 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.773135    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
May 29 17:58:56.773271 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.773187    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
May 29 17:58:56.773271 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.773214    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 29 17:58:56.774330 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.774316    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bz5c2\""
May 29 17:58:56.780757 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.780735    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"]
May 29 17:58:56.915567 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.915538    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f098669-c7a3-46ba-9cf6-421589a97e8d-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:56.915946 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.915570    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f098669-c7a3-46ba-9cf6-421589a97e8d-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:56.915946 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.915602    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f098669-c7a3-46ba-9cf6-421589a97e8d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:56.915946 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:56.915672    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cq6r\" (UniqueName: \"kubernetes.io/projected/7f098669-c7a3-46ba-9cf6-421589a97e8d-kube-api-access-5cq6r\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.016352 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.016317    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f098669-c7a3-46ba-9cf6-421589a97e8d-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.017792 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.016360    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f098669-c7a3-46ba-9cf6-421589a97e8d-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.017792 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.016729    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f098669-c7a3-46ba-9cf6-421589a97e8d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.017792 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.016773    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5cq6r\" (UniqueName: \"kubernetes.io/projected/7f098669-c7a3-46ba-9cf6-421589a97e8d-kube-api-access-5cq6r\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.017792 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:57.016905    2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
May 29 17:58:57.017792 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:58:57.016984    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f098669-c7a3-46ba-9cf6-421589a97e8d-prometheus-operator-tls podName:7f098669-c7a3-46ba-9cf6-421589a97e8d nodeName:}" failed. No retries permitted until 2026-05-29 17:58:57.516963401 +0000 UTC m=+112.572756743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7f098669-c7a3-46ba-9cf6-421589a97e8d-prometheus-operator-tls") pod "prometheus-operator-658bcd8bdf-hfjhb" (UID: "7f098669-c7a3-46ba-9cf6-421589a97e8d") : secret "prometheus-operator-tls" not found
May 29 17:58:57.017792 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.017558    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f098669-c7a3-46ba-9cf6-421589a97e8d-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.021927 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.021906    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f098669-c7a3-46ba-9cf6-421589a97e8d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.028290 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.028270    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cq6r\" (UniqueName: \"kubernetes.io/projected/7f098669-c7a3-46ba-9cf6-421589a97e8d-kube-api-access-5cq6r\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.520480 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.520445    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f098669-c7a3-46ba-9cf6-421589a97e8d-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.522819 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.522795    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f098669-c7a3-46ba-9cf6-421589a97e8d-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-hfjhb\" (UID: \"7f098669-c7a3-46ba-9cf6-421589a97e8d\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.679093 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.679049    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"
May 29 17:58:57.795508 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.795445    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb"]
May 29 17:58:57.799508 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:58:57.799482    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f098669_c7a3_46ba_9cf6_421589a97e8d.slice/crio-6b3f6094fdccc368ea91def1a88e087fd0cd956de793cdba55db055114c5b094 WatchSource:0}: Error finding container 6b3f6094fdccc368ea91def1a88e087fd0cd956de793cdba55db055114c5b094: Status 404 returned error can't find the container with id 6b3f6094fdccc368ea91def1a88e087fd0cd956de793cdba55db055114c5b094
May 29 17:58:57.890693 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:57.890666    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb" event={"ID":"7f098669-c7a3-46ba-9cf6-421589a97e8d","Type":"ContainerStarted","Data":"6b3f6094fdccc368ea91def1a88e087fd0cd956de793cdba55db055114c5b094"}
May 29 17:58:58.611628 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:58.611599    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:58:59.897487 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:59.897447    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb" event={"ID":"7f098669-c7a3-46ba-9cf6-421589a97e8d","Type":"ContainerStarted","Data":"00d18cea36f0195dfcea423283f70d37e5caed5c832f87762d6c0fcc1298fed7"}
May 29 17:58:59.897487 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:59.897488    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb" event={"ID":"7f098669-c7a3-46ba-9cf6-421589a97e8d","Type":"ContainerStarted","Data":"075a9da41c0e2fc2c7cfca592e3e4d5e60d2a1c5ce1d2d5aa2e6512c6b686b66"}
May 29 17:58:59.914048 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:58:59.914002    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-hfjhb" podStartSLOduration=2.5564663899999998 podStartE2EDuration="3.913989854s" podCreationTimestamp="2026-05-29 17:58:56 +0000 UTC" firstStartedPulling="2026-05-29 17:58:57.803006296 +0000 UTC m=+112.858799636" lastFinishedPulling="2026-05-29 17:58:59.160529751 +0000 UTC m=+114.216323100" observedRunningTime="2026-05-29 17:58:59.912714098 +0000 UTC m=+114.968507460" watchObservedRunningTime="2026-05-29 17:58:59.913989854 +0000 UTC m=+114.969783217"
May 29 17:59:02.107998 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.107969    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"]
May 29 17:59:02.111268 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.111248    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.113906 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.113886    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
May 29 17:59:02.114807 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.114789    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5xjt2\""
May 29 17:59:02.115226 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.115206    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
May 29 17:59:02.115226 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.115215    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
May 29 17:59:02.123053 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.123034    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jz8kn"]
May 29 17:59:02.126295 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.126271    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"]
May 29 17:59:02.126452 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.126438    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.129474 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.129448    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8wxqg\""
May 29 17:59:02.129474 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.129455    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 29 17:59:02.129617 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.129513    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 29 17:59:02.129617 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.129525    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 29 17:59:02.255105 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255070    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.255105 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255111    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-textfile\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.255358 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255142    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a397d4a0-d64c-452d-8e21-00e0030c3db8-root\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.255358 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255178    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-wtmp\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.255358 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255206    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a397d4a0-d64c-452d-8e21-00e0030c3db8-sys\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.255358 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255233    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.255358 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255283    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kwvz\" (UniqueName: \"kubernetes.io/projected/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-api-access-4kwvz\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.255358 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.255358 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a397d4a0-d64c-452d-8e21-00e0030c3db8-metrics-client-ca\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.255674 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255396    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d9b51e-62a0-446b-b98a-685f0144feef-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.255674 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255430    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.255674 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255473    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-tls\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.255674 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255498    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnt5b\" (UniqueName: \"kubernetes.io/projected/a397d4a0-d64c-452d-8e21-00e0030c3db8-kube-api-access-dnt5b\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.255674 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255538    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-accelerators-collector-config\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.255674 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.255566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f4d9b51e-62a0-446b-b98a-685f0144feef-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.356664 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.356631    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-accelerators-collector-config\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.356664 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.356666    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f4d9b51e-62a0-446b-b98a-685f0144feef-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.356898 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.356802    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.356898 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.356844    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-textfile\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.356898 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.356877    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a397d4a0-d64c-452d-8e21-00e0030c3db8-root\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357044 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.356902    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-wtmp\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357044 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.356940    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a397d4a0-d64c-452d-8e21-00e0030c3db8-sys\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357044 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.356993    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.357044 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.356995    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a397d4a0-d64c-452d-8e21-00e0030c3db8-root\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357047    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f4d9b51e-62a0-446b-b98a-685f0144feef-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.357264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357058    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a397d4a0-d64c-452d-8e21-00e0030c3db8-sys\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357049    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4kwvz\" (UniqueName: \"kubernetes.io/projected/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-api-access-4kwvz\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.357264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357105    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-wtmp\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357131    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.357264 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:59:02.357145    2575 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
May 29 17:59:02.357264 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:59:02.357228    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-tls podName:f4d9b51e-62a0-446b-b98a-685f0144feef nodeName:}" failed. No retries permitted until 2026-05-29 17:59:02.857209075 +0000 UTC m=+117.913002433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-tls") pod "kube-state-metrics-8469c486bb-jxvll" (UID: "f4d9b51e-62a0-446b-b98a-685f0144feef") : secret "kube-state-metrics-tls" not found
May 29 17:59:02.357264 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357230    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-textfile\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357662 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357323    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-accelerators-collector-config\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357662 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357401    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a397d4a0-d64c-452d-8e21-00e0030c3db8-metrics-client-ca\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357662 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357466    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d9b51e-62a0-446b-b98a-685f0144feef-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.357662 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357503    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357662 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357556    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-tls\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357662 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357581    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dnt5b\" (UniqueName: \"kubernetes.io/projected/a397d4a0-d64c-452d-8e21-00e0030c3db8-kube-api-access-dnt5b\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.357970 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:59:02.357671    2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 29 17:59:02.357970 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:59:02.357715    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-tls podName:a397d4a0-d64c-452d-8e21-00e0030c3db8 nodeName:}" failed. No retries permitted until 2026-05-29 17:59:02.85770232 +0000 UTC m=+117.913495673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-tls") pod "node-exporter-jz8kn" (UID: "a397d4a0-d64c-452d-8e21-00e0030c3db8") : secret "node-exporter-tls" not found
May 29 17:59:02.357970 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357800    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.357970 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.357958    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a397d4a0-d64c-452d-8e21-00e0030c3db8-metrics-client-ca\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.358133 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.358051    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d9b51e-62a0-446b-b98a-685f0144feef-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.359450 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.359431    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.359782 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.359767    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.365926 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.365903    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnt5b\" (UniqueName: \"kubernetes.io/projected/a397d4a0-d64c-452d-8e21-00e0030c3db8-kube-api-access-dnt5b\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.366062 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.365903    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kwvz\" (UniqueName: \"kubernetes.io/projected/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-api-access-4kwvz\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.861878 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.861843    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-tls\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:02.862071 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.861930    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:02.862071 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:59:02.862065    2575 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
May 29 17:59:02.862215 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:59:02.862136    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-tls podName:f4d9b51e-62a0-446b-b98a-685f0144feef nodeName:}" failed. No retries permitted until 2026-05-29 17:59:03.862116736 +0000 UTC m=+118.917910077 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-tls") pod "kube-state-metrics-8469c486bb-jxvll" (UID: "f4d9b51e-62a0-446b-b98a-685f0144feef") : secret "kube-state-metrics-tls" not found
May 29 17:59:02.864293 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:02.864267    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a397d4a0-d64c-452d-8e21-00e0030c3db8-node-exporter-tls\") pod \"node-exporter-jz8kn\" (UID: \"a397d4a0-d64c-452d-8e21-00e0030c3db8\") " pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:03.038625 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.038593    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jz8kn"
May 29 17:59:03.049679 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:59:03.049634    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda397d4a0_d64c_452d_8e21_00e0030c3db8.slice/crio-0d9751696218c28a17122245866de202e670a98255a13238df51371a5f11b3ae WatchSource:0}: Error finding container 0d9751696218c28a17122245866de202e670a98255a13238df51371a5f11b3ae: Status 404 returned error can't find the container with id 0d9751696218c28a17122245866de202e670a98255a13238df51371a5f11b3ae
May 29 17:59:03.152704 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.152671    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 17:59:03.157829 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.157798    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.160359 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.160333    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 29 17:59:03.160498 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.160368    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 29 17:59:03.160498 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.160452    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 29 17:59:03.160666 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.160648    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 29 17:59:03.160961 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.160802    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 29 17:59:03.160961 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.160832    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 29 17:59:03.160961 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.160842    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h6986\""
May 29 17:59:03.160961 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.160849    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 29 17:59:03.160961 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.160884    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 29 17:59:03.160961 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.160949    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 29 17:59:03.168574 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.168552    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 17:59:03.265249 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265219    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265249 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265254    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-web-config\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265483 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265279    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265483 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265302    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265483 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265348    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-out\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265483 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265384    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265616 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265480    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2pz7\" (UniqueName: \"kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-kube-api-access-c2pz7\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265616 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265531    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265616 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265567    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265616 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265592    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265732 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265664    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265732 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265691    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.265796 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.265747    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-volume\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.366855 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.366818    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367031 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.366864    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367031 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.366907    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-volume\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367031 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.366952    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367031 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.366977    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-web-config\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367031 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.367002    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367031 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.367027    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367407 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.367054    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-out\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367407 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.367091    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367407 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.367138    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c2pz7\" (UniqueName: \"kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-kube-api-access-c2pz7\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367407 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.367188    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367407 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.367211    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.367407 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.367237    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.368622 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.368296    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.368622 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.368503    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.368854 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.368825    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.371357 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.371316    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-out\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.371620 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.371595    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.372091 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.372068    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.372557 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.372482    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-volume\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.372557 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.372527    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-web-config\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.372691 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.372591    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.373000 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.372982    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.373318 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.373298    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.373596 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.373577    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.376434 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.376414    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2pz7\" (UniqueName: \"kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-kube-api-access-c2pz7\") pod \"alertmanager-main-0\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.467900 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.467820    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 17:59:03.598510 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.598367    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 17:59:03.602291 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:59:03.602264    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod948ca724_ab0b_487d_875c_b8de8ba1a95b.slice/crio-d98c9d350f1e3eea2c3ba340062e432917528630283ac47c3f627cf074176dcd WatchSource:0}: Error finding container d98c9d350f1e3eea2c3ba340062e432917528630283ac47c3f627cf074176dcd: Status 404 returned error can't find the container with id d98c9d350f1e3eea2c3ba340062e432917528630283ac47c3f627cf074176dcd
May 29 17:59:03.871983 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.871943    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:03.874593 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.874568    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4d9b51e-62a0-446b-b98a-685f0144feef-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jxvll\" (UID: \"f4d9b51e-62a0-446b-b98a-685f0144feef\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:03.908807 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.908755    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerStarted","Data":"d98c9d350f1e3eea2c3ba340062e432917528630283ac47c3f627cf074176dcd"}
May 29 17:59:03.909886 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.909864    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jz8kn" event={"ID":"a397d4a0-d64c-452d-8e21-00e0030c3db8","Type":"ContainerStarted","Data":"0d9751696218c28a17122245866de202e670a98255a13238df51371a5f11b3ae"}
May 29 17:59:03.920220 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:03.920199    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"
May 29 17:59:04.178078 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:04.178049    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-jxvll"]
May 29 17:59:04.183981 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:59:04.183953    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d9b51e_62a0_446b_b98a_685f0144feef.slice/crio-c9f36c1d52891172fc8fe5fcbd8951f06d599cac056b518fff14593b9cd26876 WatchSource:0}: Error finding container c9f36c1d52891172fc8fe5fcbd8951f06d599cac056b518fff14593b9cd26876: Status 404 returned error can't find the container with id c9f36c1d52891172fc8fe5fcbd8951f06d599cac056b518fff14593b9cd26876
May 29 17:59:04.914062 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:04.914033    2575 generic.go:358] "Generic (PLEG): container finished" podID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerID="8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6" exitCode=0
May 29 17:59:04.914196 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:04.914112    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerDied","Data":"8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6"}
May 29 17:59:04.915698 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:04.915669    2575 generic.go:358] "Generic (PLEG): container finished" podID="a397d4a0-d64c-452d-8e21-00e0030c3db8" containerID="e8001e6b41052ffdcf57dbf5ea3e9f4c0c38b31f97c26229f51396788f21b5f9" exitCode=0
May 29 17:59:04.915770 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:04.915726    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jz8kn" event={"ID":"a397d4a0-d64c-452d-8e21-00e0030c3db8","Type":"ContainerDied","Data":"e8001e6b41052ffdcf57dbf5ea3e9f4c0c38b31f97c26229f51396788f21b5f9"}
May 29 17:59:04.916905 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:04.916823    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll" event={"ID":"f4d9b51e-62a0-446b-b98a-685f0144feef","Type":"ContainerStarted","Data":"c9f36c1d52891172fc8fe5fcbd8951f06d599cac056b518fff14593b9cd26876"}
May 29 17:59:05.194713 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.194674    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"]
May 29 17:59:05.198613 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.198588    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.201736 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.201433    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
May 29 17:59:05.201736 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.201471    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
May 29 17:59:05.201736 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.201528    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
May 29 17:59:05.201736 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.201578    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
May 29 17:59:05.201736 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.201579    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
May 29 17:59:05.201736 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.201531    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h69tf\""
May 29 17:59:05.201736 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.201671    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bnpetcjh82h0d\""
May 29 17:59:05.210372 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.210352    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"]
May 29 17:59:05.284625 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.284592    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.284625 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.284629    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxf9h\" (UniqueName: \"kubernetes.io/projected/9a1ee7c9-d367-4af4-8801-f4e708e62eed-kube-api-access-lxf9h\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.284897 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.284673    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.284897 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.284756    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-tls\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.284897 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.284785    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-grpc-tls\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.285046 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.284888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.285046 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.284933    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a1ee7c9-d367-4af4-8801-f4e708e62eed-metrics-client-ca\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.285046 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.284967    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.386353 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.386318    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.386515 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.386378    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-tls\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.386515 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.386406    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-grpc-tls\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.386515 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.386469    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.386515 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.386496    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a1ee7c9-d367-4af4-8801-f4e708e62eed-metrics-client-ca\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.386749 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.386525    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.386749 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.386577    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.386749 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.386605    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lxf9h\" (UniqueName: \"kubernetes.io/projected/9a1ee7c9-d367-4af4-8801-f4e708e62eed-kube-api-access-lxf9h\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.387252 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.387230    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a1ee7c9-d367-4af4-8801-f4e708e62eed-metrics-client-ca\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.389168 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.389130    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
May 29 17:59:05.389268 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.389187    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
May 29 17:59:05.389268 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.389240    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
May 29 17:59:05.389397 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.389347    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
May 29 17:59:05.389397 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.389389    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bnpetcjh82h0d\""
May 29 17:59:05.389564 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.389457    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
May 29 17:59:05.395775 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.395749    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxf9h\" (UniqueName: \"kubernetes.io/projected/9a1ee7c9-d367-4af4-8801-f4e708e62eed-kube-api-access-lxf9h\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.399204 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.399179    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.400030 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.400006    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.400124 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.400011    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.400303 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.400262    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-grpc-tls\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.400601 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.400580    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.401488 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.401469    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9a1ee7c9-d367-4af4-8801-f4e708e62eed-secret-thanos-querier-tls\") pod \"thanos-querier-648d5c57bb-l9d6d\" (UID: \"9a1ee7c9-d367-4af4-8801-f4e708e62eed\") " pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.512807 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.512763    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h69tf\""
May 29 17:59:05.521305 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.521221    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:05.921009 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.920978    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"]
May 29 17:59:05.922822 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.922799    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jz8kn" event={"ID":"a397d4a0-d64c-452d-8e21-00e0030c3db8","Type":"ContainerStarted","Data":"356f873956bd98bd76a3a1139d2f301e36a5ffa31be885acf2e258c219857b95"}
May 29 17:59:05.922929 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.922832    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jz8kn" event={"ID":"a397d4a0-d64c-452d-8e21-00e0030c3db8","Type":"ContainerStarted","Data":"9a1eb8b957e00f19a1250a775c9781fb2ab4a616479cb69b5c33cdd1be81fda2"}
May 29 17:59:05.924689 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.924660    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll" event={"ID":"f4d9b51e-62a0-446b-b98a-685f0144feef","Type":"ContainerStarted","Data":"6f576f88b59fcf6a7b86026ffc3729dd02715c2f0558166f93435139e4874d6c"}
May 29 17:59:05.943095 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:05.943032    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jz8kn" podStartSLOduration=2.900415777 podStartE2EDuration="3.943013978s" podCreationTimestamp="2026-05-29 17:59:02 +0000 UTC" firstStartedPulling="2026-05-29 17:59:03.051787367 +0000 UTC m=+118.107580721" lastFinishedPulling="2026-05-29 17:59:04.094385575 +0000 UTC m=+119.150178922" observedRunningTime="2026-05-29 17:59:05.941728635 +0000 UTC m=+120.997522000" watchObservedRunningTime="2026-05-29 17:59:05.943013978 +0000 UTC m=+120.998807341"
May 29 17:59:06.275839 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:59:06.275733    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1ee7c9_d367_4af4_8801_f4e708e62eed.slice/crio-391c6ea8887ce26fb55cd1b8fe8a31d84b2ee1b7b04b2b960872d108d172ff3e WatchSource:0}: Error finding container 391c6ea8887ce26fb55cd1b8fe8a31d84b2ee1b7b04b2b960872d108d172ff3e: Status 404 returned error can't find the container with id 391c6ea8887ce26fb55cd1b8fe8a31d84b2ee1b7b04b2b960872d108d172ff3e
May 29 17:59:06.553557 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.553525    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-746d756ffc-l9sj2"]
May 29 17:59:06.558260 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.558232    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.561006 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.560984    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 29 17:59:06.561138 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.561116    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7innkc45jct6g\""
May 29 17:59:06.561513 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.561295    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
May 29 17:59:06.561513 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.561315    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-sdjtl\""
May 29 17:59:06.561513 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.561431    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
May 29 17:59:06.562097 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.562019    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
May 29 17:59:06.569516 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.569484    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-746d756ffc-l9sj2"]
May 29 17:59:06.598018 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.597991    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b8d54a26-2978-440c-8d64-f037eb2eff84-metrics-server-audit-profiles\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.598124 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.598045    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b8d54a26-2978-440c-8d64-f037eb2eff84-client-ca-bundle\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.598124 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.598109    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b8d54a26-2978-440c-8d64-f037eb2eff84-audit-log\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.598301 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.598180    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b8d54a26-2978-440c-8d64-f037eb2eff84-secret-metrics-server-client-certs\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.598301 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.598229    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzr7d\" (UniqueName: \"kubernetes.io/projected/b8d54a26-2978-440c-8d64-f037eb2eff84-kube-api-access-pzr7d\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.598301 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.598280    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b8d54a26-2978-440c-8d64-f037eb2eff84-secret-metrics-server-tls\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.598450 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.598326    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8d54a26-2978-440c-8d64-f037eb2eff84-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.699476 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.699443    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b8d54a26-2978-440c-8d64-f037eb2eff84-secret-metrics-server-tls\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.699601 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.699484    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8d54a26-2978-440c-8d64-f037eb2eff84-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.699601 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.699554    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b8d54a26-2978-440c-8d64-f037eb2eff84-metrics-server-audit-profiles\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.699601 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.699587    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b8d54a26-2978-440c-8d64-f037eb2eff84-client-ca-bundle\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.699768 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.699744    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b8d54a26-2978-440c-8d64-f037eb2eff84-audit-log\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.699821 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.699796    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b8d54a26-2978-440c-8d64-f037eb2eff84-secret-metrics-server-client-certs\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.699878 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.699831    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pzr7d\" (UniqueName: \"kubernetes.io/projected/b8d54a26-2978-440c-8d64-f037eb2eff84-kube-api-access-pzr7d\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.700225 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.700146    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b8d54a26-2978-440c-8d64-f037eb2eff84-audit-log\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.700433 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.700406    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8d54a26-2978-440c-8d64-f037eb2eff84-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.700754 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.700732    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b8d54a26-2978-440c-8d64-f037eb2eff84-metrics-server-audit-profiles\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.701884 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.701867    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b8d54a26-2978-440c-8d64-f037eb2eff84-client-ca-bundle\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.701955 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.701939    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b8d54a26-2978-440c-8d64-f037eb2eff84-secret-metrics-server-tls\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.702080 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.702063    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b8d54a26-2978-440c-8d64-f037eb2eff84-secret-metrics-server-client-certs\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.707902 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.707867    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzr7d\" (UniqueName: \"kubernetes.io/projected/b8d54a26-2978-440c-8d64-f037eb2eff84-kube-api-access-pzr7d\") pod \"metrics-server-746d756ffc-l9sj2\" (UID: \"b8d54a26-2978-440c-8d64-f037eb2eff84\") " pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.879551 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.879517    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:06.902922 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.900779    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"]
May 29 17:59:06.908234 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.908148    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"
May 29 17:59:06.909708 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.909678    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"]
May 29 17:59:06.911408 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.911387    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
May 29 17:59:06.911813 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.911626    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-8dfcc\""
May 29 17:59:06.934048 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.934011    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d" event={"ID":"9a1ee7c9-d367-4af4-8801-f4e708e62eed","Type":"ContainerStarted","Data":"391c6ea8887ce26fb55cd1b8fe8a31d84b2ee1b7b04b2b960872d108d172ff3e"}
May 29 17:59:06.938662 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.938602    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerStarted","Data":"6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20"}
May 29 17:59:06.938795 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.938639    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerStarted","Data":"b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8"}
May 29 17:59:06.938795 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.938686    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerStarted","Data":"4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179"}
May 29 17:59:06.938795 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.938699    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerStarted","Data":"c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0"}
May 29 17:59:06.938795 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.938711    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerStarted","Data":"d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7"}
May 29 17:59:06.940945 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.940874    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll" event={"ID":"f4d9b51e-62a0-446b-b98a-685f0144feef","Type":"ContainerStarted","Data":"a1b8d0695d9597c3cbdd1cd5a5879e4925ede050eeaaa5f3c58b0613c7a84d7b"}
May 29 17:59:06.940945 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.940903    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll" event={"ID":"f4d9b51e-62a0-446b-b98a-685f0144feef","Type":"ContainerStarted","Data":"a2ca4fb7adb9f916639ccdabd5809e7c45c2a6a08a5cf8d47a10dcd957a146ea"}
May 29 17:59:06.964228 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:06.964147    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jxvll" podStartSLOduration=3.3199449420000002 podStartE2EDuration="4.964126905s" podCreationTimestamp="2026-05-29 17:59:02 +0000 UTC" firstStartedPulling="2026-05-29 17:59:04.186138023 +0000 UTC m=+119.241931362" lastFinishedPulling="2026-05-29 17:59:05.830319985 +0000 UTC m=+120.886113325" observedRunningTime="2026-05-29 17:59:06.963779476 +0000 UTC m=+122.019572835" watchObservedRunningTime="2026-05-29 17:59:06.964126905 +0000 UTC m=+122.019920268"
May 29 17:59:07.005768 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.005732    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99c5820d-68c6-4645-9daf-4a1a62155af8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-brfsr\" (UID: \"99c5820d-68c6-4645-9daf-4a1a62155af8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"
May 29 17:59:07.028327 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.028183    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-746d756ffc-l9sj2"]
May 29 17:59:07.032191 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:59:07.032140    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d54a26_2978_440c_8d64_f037eb2eff84.slice/crio-ac687027d357e3961077605205a4039dc7f1a3dd0cfc1914c258a0cbdf720d3c WatchSource:0}: Error finding container ac687027d357e3961077605205a4039dc7f1a3dd0cfc1914c258a0cbdf720d3c: Status 404 returned error can't find the container with id ac687027d357e3961077605205a4039dc7f1a3dd0cfc1914c258a0cbdf720d3c
May 29 17:59:07.107297 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.107266    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99c5820d-68c6-4645-9daf-4a1a62155af8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-brfsr\" (UID: \"99c5820d-68c6-4645-9daf-4a1a62155af8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"
May 29 17:59:07.107468 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:59:07.107438    2575 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
May 29 17:59:07.107600 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:59:07.107520    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99c5820d-68c6-4645-9daf-4a1a62155af8-monitoring-plugin-cert podName:99c5820d-68c6-4645-9daf-4a1a62155af8 nodeName:}" failed. No retries permitted until 2026-05-29 17:59:07.607500176 +0000 UTC m=+122.663293529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/99c5820d-68c6-4645-9daf-4a1a62155af8-monitoring-plugin-cert") pod "monitoring-plugin-5768fcc9dd-brfsr" (UID: "99c5820d-68c6-4645-9daf-4a1a62155af8") : secret "monitoring-plugin-cert" not found
May 29 17:59:07.323799 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.323650    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"]
May 29 17:59:07.327308 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.327287    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.330333 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.330135    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
May 29 17:59:07.330333 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.330180    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
May 29 17:59:07.330615 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.330459    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
May 29 17:59:07.331270 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.331098    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-cnpwk\""
May 29 17:59:07.331522 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.331397    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
May 29 17:59:07.332050 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.332006    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
May 29 17:59:07.335920 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.335819    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
May 29 17:59:07.345238 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.345213    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"]
May 29 17:59:07.411400 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.410952    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-secret-telemeter-client\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.411400 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.411018    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.411400 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.411048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-federate-client-tls\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.411400 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.411073    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/820f9f0a-a2f7-4418-b28c-ec862a52fe68-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.411400 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.411121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg8hm\" (UniqueName: \"kubernetes.io/projected/820f9f0a-a2f7-4418-b28c-ec862a52fe68-kube-api-access-bg8hm\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.411400 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.411194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/820f9f0a-a2f7-4418-b28c-ec862a52fe68-metrics-client-ca\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.411400 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.411281    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/820f9f0a-a2f7-4418-b28c-ec862a52fe68-serving-certs-ca-bundle\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.411400 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.411310    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-telemeter-client-tls\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.511912 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.511879    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-telemeter-client-tls\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.512073 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.511933    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-secret-telemeter-client\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.512073 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.511981    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.512073 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.512013    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-federate-client-tls\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.512073 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.512039    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/820f9f0a-a2f7-4418-b28c-ec862a52fe68-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.512339 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.512079    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bg8hm\" (UniqueName: \"kubernetes.io/projected/820f9f0a-a2f7-4418-b28c-ec862a52fe68-kube-api-access-bg8hm\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.512339 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.512141    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/820f9f0a-a2f7-4418-b28c-ec862a52fe68-metrics-client-ca\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.512339 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.512255    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/820f9f0a-a2f7-4418-b28c-ec862a52fe68-serving-certs-ca-bundle\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.513069 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.513008    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/820f9f0a-a2f7-4418-b28c-ec862a52fe68-serving-certs-ca-bundle\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.513198 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.513137    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/820f9f0a-a2f7-4418-b28c-ec862a52fe68-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.513281 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.513202    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/820f9f0a-a2f7-4418-b28c-ec862a52fe68-metrics-client-ca\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.515213 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.515188    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-federate-client-tls\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.515575 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.515530    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-secret-telemeter-client\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.515927 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.515901    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.516402 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.516361    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/820f9f0a-a2f7-4418-b28c-ec862a52fe68-telemeter-client-tls\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.521791 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.521752    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg8hm\" (UniqueName: \"kubernetes.io/projected/820f9f0a-a2f7-4418-b28c-ec862a52fe68-kube-api-access-bg8hm\") pod \"telemeter-client-5cb66f89fc-scjh2\" (UID: \"820f9f0a-a2f7-4418-b28c-ec862a52fe68\") " pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.613088 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.612979    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99c5820d-68c6-4645-9daf-4a1a62155af8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-brfsr\" (UID: \"99c5820d-68c6-4645-9daf-4a1a62155af8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"
May 29 17:59:07.615961 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.615935    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/99c5820d-68c6-4645-9daf-4a1a62155af8-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-brfsr\" (UID: \"99c5820d-68c6-4645-9daf-4a1a62155af8\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"
May 29 17:59:07.639228 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.639148    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"
May 29 17:59:07.820072 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.820034    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"
May 29 17:59:07.945319 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:07.945281    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2" event={"ID":"b8d54a26-2978-440c-8d64-f037eb2eff84","Type":"ContainerStarted","Data":"ac687027d357e3961077605205a4039dc7f1a3dd0cfc1914c258a0cbdf720d3c"}
May 29 17:59:08.360531 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.360493    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 17:59:08.365346 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.365319    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.369744 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.369577    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9sjkecoht317d\""
May 29 17:59:08.369744 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.369617    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 29 17:59:08.369744 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.369669    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 29 17:59:08.369976 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.369763    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 29 17:59:08.369976 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.369848    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 29 17:59:08.370198 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.370179    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 29 17:59:08.371319 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.370905    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 29 17:59:08.371319 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.371249    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 29 17:59:08.371463 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.371357    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 29 17:59:08.371463 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.371360    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vbdnz\""
May 29 17:59:08.371463 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.371405    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 29 17:59:08.371463 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.371437    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 29 17:59:08.375768 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.373680    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 29 17:59:08.383653 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.383625    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 29 17:59:08.393127 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.393029    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 17:59:08.422471 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422433    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.422642 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422562    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.422642 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422598    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.422642 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422626    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.422764 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.422764 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422735    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.422846 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422770    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.422846 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422814    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.422933 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422882    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.422985 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422939    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-web-config\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.423021 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.422997    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.423055 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.423029    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.423087 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.423063    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.423124 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.423112    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config-out\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.423196 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.423143    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.423196 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.423186    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.423276 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.423207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.423276 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.423224    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rm6c\" (UniqueName: \"kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-kube-api-access-7rm6c\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.524467 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524363    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.524467 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524416    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.524467 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524440    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.524744 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524465    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.524744 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524583    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.524744 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524615    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.524744 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524659    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.524744 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524697    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.524744 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524725    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-web-config\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.525026 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524770    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.525026 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524805    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.525026 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524841    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.525026 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524875    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config-out\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.525026 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524914    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.525026 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524944    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.525026 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.524976    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.525026 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.525000    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7rm6c\" (UniqueName: \"kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-kube-api-access-7rm6c\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.525612 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.525027    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.525612 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.525279    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.526694 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.526663    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.527250 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.527209    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.527411 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.527281    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.529293 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.528384    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.529293 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.528674    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.529293 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.529006    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.532359 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.531223    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.532359 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.531590    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.533209 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.532998    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.533557 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.533515    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.533643 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.533576    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.533910 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.533837    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.533910 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.533892    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-web-config\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.534473 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.534433    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.535447 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.535426    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config-out\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.536045 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.536008    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.539391 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.539367    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rm6c\" (UniqueName: \"kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-kube-api-access-7rm6c\") pod \"prometheus-k8s-0\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.682250 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.682216    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:08.958104 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.958037    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d" event={"ID":"9a1ee7c9-d367-4af4-8801-f4e708e62eed","Type":"ContainerStarted","Data":"fe139a5f382bf52d69aae8b17b0e786cb3a99d273f5058e190e5b6dd44c4d83d"}
May 29 17:59:08.960368 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.960319    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2" event={"ID":"b8d54a26-2978-440c-8d64-f037eb2eff84","Type":"ContainerStarted","Data":"82dc56b46d6ad56e40e01db44dd9d0c165174332557f1363185a4520c2a108fc"}
May 29 17:59:08.965941 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.964939    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerStarted","Data":"79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b"}
May 29 17:59:08.978895 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.978844    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"]
May 29 17:59:08.995783 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:08.995689    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2" podStartSLOduration=1.271009737 podStartE2EDuration="2.99567159s" podCreationTimestamp="2026-05-29 17:59:06 +0000 UTC" firstStartedPulling="2026-05-29 17:59:07.034576345 +0000 UTC m=+122.090369689" lastFinishedPulling="2026-05-29 17:59:08.759238189 +0000 UTC m=+123.815031542" observedRunningTime="2026-05-29 17:59:08.995620397 +0000 UTC m=+124.051413760" watchObservedRunningTime="2026-05-29 17:59:08.99567159 +0000 UTC m=+124.051464953"
May 29 17:59:09.015099 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.015011    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cb66f89fc-scjh2"]
May 29 17:59:09.054020 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.053952    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 17:59:09.055409 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.055357    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=0.923436749 podStartE2EDuration="6.055337908s" podCreationTimestamp="2026-05-29 17:59:03 +0000 UTC" firstStartedPulling="2026-05-29 17:59:03.604537024 +0000 UTC m=+118.660330368" lastFinishedPulling="2026-05-29 17:59:08.736438174 +0000 UTC m=+123.792231527" observedRunningTime="2026-05-29 17:59:09.054389891 +0000 UTC m=+124.110183311" watchObservedRunningTime="2026-05-29 17:59:09.055337908 +0000 UTC m=+124.111131271"
May 29 17:59:09.061832 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:59:09.061805    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958a3e9c_5087_4e41_a4c1_e372c6de91ee.slice/crio-55309edf2179ae5b88ed34ccbb8934e17a63a403a399444687212f46864ea245 WatchSource:0}: Error finding container 55309edf2179ae5b88ed34ccbb8934e17a63a403a399444687212f46864ea245: Status 404 returned error can't find the container with id 55309edf2179ae5b88ed34ccbb8934e17a63a403a399444687212f46864ea245
May 29 17:59:09.969690 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.969650    2575 generic.go:358] "Generic (PLEG): container finished" podID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerID="fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144" exitCode=0
May 29 17:59:09.970136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.969737    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerDied","Data":"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"}
May 29 17:59:09.970136 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.969770    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerStarted","Data":"55309edf2179ae5b88ed34ccbb8934e17a63a403a399444687212f46864ea245"}
May 29 17:59:09.971478 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.971438    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2" event={"ID":"820f9f0a-a2f7-4418-b28c-ec862a52fe68","Type":"ContainerStarted","Data":"31f87e9e40a58fe7b546e5b852bf75945a1db2caa48886f97bb6ed886e89ba21"}
May 29 17:59:09.974742 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.974717    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d" event={"ID":"9a1ee7c9-d367-4af4-8801-f4e708e62eed","Type":"ContainerStarted","Data":"36c1fccb2e83d6dad1ace84f896f4649c2b61e8c54b4af4c7627e842bef56b33"}
May 29 17:59:09.974854 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.974749    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d" event={"ID":"9a1ee7c9-d367-4af4-8801-f4e708e62eed","Type":"ContainerStarted","Data":"8007d90d9179e76ce3e3aedba9fcfb12d987d9aed0a4dcadd52b7e19dfad736b"}
May 29 17:59:09.974854 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.974762    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d" event={"ID":"9a1ee7c9-d367-4af4-8801-f4e708e62eed","Type":"ContainerStarted","Data":"2229f600d4b09f77c020d0d48410916327cae978b82a215f2dd4a27b77668b2a"}
May 29 17:59:09.974854 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.974774    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d" event={"ID":"9a1ee7c9-d367-4af4-8801-f4e708e62eed","Type":"ContainerStarted","Data":"aca3acd54641e165fa2b0dfd3f2491f02402303b5dbcf925a3ebc68b94541e1d"}
May 29 17:59:09.974854 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.974786    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d" event={"ID":"9a1ee7c9-d367-4af4-8801-f4e708e62eed","Type":"ContainerStarted","Data":"c912bbfa966d0e295fb1087843cfe58df67ccf3e09a42c64f50c98772cec14a0"}
May 29 17:59:09.975048 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.974905    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:09.975955 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:09.975909    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr" event={"ID":"99c5820d-68c6-4645-9daf-4a1a62155af8","Type":"ContainerStarted","Data":"b135b59e699643bf97f45cf2d8e5ae9627fcd8ae4de29cb512333274ce181ca6"}
May 29 17:59:10.021115 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:10.021063    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d" podStartSLOduration=2.538425543 podStartE2EDuration="5.021049661s" podCreationTimestamp="2026-05-29 17:59:05 +0000 UTC" firstStartedPulling="2026-05-29 17:59:06.277445286 +0000 UTC m=+121.333238625" lastFinishedPulling="2026-05-29 17:59:08.760069396 +0000 UTC m=+123.815862743" observedRunningTime="2026-05-29 17:59:10.019367881 +0000 UTC m=+125.075161247" watchObservedRunningTime="2026-05-29 17:59:10.021049661 +0000 UTC m=+125.076843022"
May 29 17:59:10.981474 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:10.981419    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr" event={"ID":"99c5820d-68c6-4645-9daf-4a1a62155af8","Type":"ContainerStarted","Data":"fd22803ee1406d48cf10cf6f223fbf10110b148a9ac6aeb09ee4590d26426647"}
May 29 17:59:10.981914 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:10.981717    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"
May 29 17:59:10.987609 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:10.987583    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr"
May 29 17:59:11.005356 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:11.005187    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-brfsr" podStartSLOduration=3.3871083840000002 podStartE2EDuration="5.005146402s" podCreationTimestamp="2026-05-29 17:59:06 +0000 UTC" firstStartedPulling="2026-05-29 17:59:08.99682217 +0000 UTC m=+124.052615523" lastFinishedPulling="2026-05-29 17:59:10.614860195 +0000 UTC m=+125.670653541" observedRunningTime="2026-05-29 17:59:11.00412379 +0000 UTC m=+126.059917153" watchObservedRunningTime="2026-05-29 17:59:11.005146402 +0000 UTC m=+126.060939764"
May 29 17:59:11.987175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:11.987097    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2" event={"ID":"820f9f0a-a2f7-4418-b28c-ec862a52fe68","Type":"ContainerStarted","Data":"c754b6594ffe3dd4d55338026d4288f40ee305e58b118a040c3d8b03e1275de0"}
May 29 17:59:11.987175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:11.987135    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2" event={"ID":"820f9f0a-a2f7-4418-b28c-ec862a52fe68","Type":"ContainerStarted","Data":"1937bc8567382bf1237207576985664b3eb07b156927e4a4a2ed98e99f03e6d2"}
May 29 17:59:11.987175 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:11.987146    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2" event={"ID":"820f9f0a-a2f7-4418-b28c-ec862a52fe68","Type":"ContainerStarted","Data":"0d5b963d1ab544ff0ef461e78496cb8195560eca09623e8fe8638fb2da05a9df"}
May 29 17:59:12.023256 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:12.023197    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5cb66f89fc-scjh2" podStartSLOduration=2.620923602 podStartE2EDuration="5.023182709s" podCreationTimestamp="2026-05-29 17:59:07 +0000 UTC" firstStartedPulling="2026-05-29 17:59:09.029269158 +0000 UTC m=+124.085062502" lastFinishedPulling="2026-05-29 17:59:11.431528248 +0000 UTC m=+126.487321609" observedRunningTime="2026-05-29 17:59:12.021756954 +0000 UTC m=+127.077550318" watchObservedRunningTime="2026-05-29 17:59:12.023182709 +0000 UTC m=+127.078976071"
May 29 17:59:13.625498 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.625461    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m" podUID="4aac1a9f-6772-4611-bc33-ac522d5de5d1" containerName="registry" containerID="cri-o://7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333" gracePeriod=30
May 29 17:59:13.863802 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.863777    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:59:13.984273 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.984187    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-installation-pull-secrets\") pod \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") "
May 29 17:59:13.984273 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.984257    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-trusted-ca\") pod \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") "
May 29 17:59:13.984468 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.984287    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-certificates\") pod \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") "
May 29 17:59:13.984468 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.984318    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjbr4\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-kube-api-access-mjbr4\") pod \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") "
May 29 17:59:13.984468 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.984340    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-image-registry-private-configuration\") pod \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") "
May 29 17:59:13.984686 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.984492    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls\") pod \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") "
May 29 17:59:13.984686 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.984545    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-bound-sa-token\") pod \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") "
May 29 17:59:13.984686 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.984581    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aac1a9f-6772-4611-bc33-ac522d5de5d1-ca-trust-extracted\") pod \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\" (UID: \"4aac1a9f-6772-4611-bc33-ac522d5de5d1\") "
May 29 17:59:13.984846 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.984795    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4aac1a9f-6772-4611-bc33-ac522d5de5d1" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 17:59:13.985064 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.985021    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4aac1a9f-6772-4611-bc33-ac522d5de5d1" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 17:59:13.986902 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.986713    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4aac1a9f-6772-4611-bc33-ac522d5de5d1" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 17:59:13.986902 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.986855    2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-certificates\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 17:59:13.986902 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.986870    2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-installation-pull-secrets\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 17:59:13.986902 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.986886    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aac1a9f-6772-4611-bc33-ac522d5de5d1-trusted-ca\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 17:59:13.986902 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.986893    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-kube-api-access-mjbr4" (OuterVolumeSpecName: "kube-api-access-mjbr4") pod "4aac1a9f-6772-4611-bc33-ac522d5de5d1" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1"). InnerVolumeSpecName "kube-api-access-mjbr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 17:59:13.987250 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.987046    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4aac1a9f-6772-4611-bc33-ac522d5de5d1" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 17:59:13.987635 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.987609    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4aac1a9f-6772-4611-bc33-ac522d5de5d1" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 17:59:13.987736 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.987712    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4aac1a9f-6772-4611-bc33-ac522d5de5d1" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 17:59:13.995043 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.995012    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aac1a9f-6772-4611-bc33-ac522d5de5d1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4aac1a9f-6772-4611-bc33-ac522d5de5d1" (UID: "4aac1a9f-6772-4611-bc33-ac522d5de5d1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 17:59:13.999415 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.999391    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerStarted","Data":"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"}
May 29 17:59:13.999500 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.999422    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerStarted","Data":"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"}
May 29 17:59:13.999500 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.999434    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerStarted","Data":"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"}
May 29 17:59:13.999500 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.999443    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerStarted","Data":"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"}
May 29 17:59:13.999500 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.999451    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerStarted","Data":"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"}
May 29 17:59:13.999500 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:13.999459    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerStarted","Data":"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"}
May 29 17:59:14.000456 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.000435    2575 generic.go:358] "Generic (PLEG): container finished" podID="4aac1a9f-6772-4611-bc33-ac522d5de5d1" containerID="7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333" exitCode=0
May 29 17:59:14.000526 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.000484    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m" event={"ID":"4aac1a9f-6772-4611-bc33-ac522d5de5d1","Type":"ContainerDied","Data":"7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333"}
May 29 17:59:14.000526 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.000499    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m"
May 29 17:59:14.000526 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.000506    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77c8d57b45-7zx6m" event={"ID":"4aac1a9f-6772-4611-bc33-ac522d5de5d1","Type":"ContainerDied","Data":"2e30430b76abbd665a3620d5b8d877827de0bf424881077ab6ab7aa628b90d9a"}
May 29 17:59:14.000526 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.000521    2575 scope.go:117] "RemoveContainer" containerID="7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333"
May 29 17:59:14.009417 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.009400    2575 scope.go:117] "RemoveContainer" containerID="7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333"
May 29 17:59:14.009685 ip-10-0-133-167 kubenswrapper[2575]: E0529 17:59:14.009667    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333\": container with ID starting with 7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333 not found: ID does not exist" containerID="7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333"
May 29 17:59:14.009733 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.009692    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333"} err="failed to get container status \"7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333\": rpc error: code = NotFound desc = could not find container \"7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333\": container with ID starting with 7db01011e47bd2da1f33aab927bb8a8b593a9464962fc2357beb29e182bf2333 not found: ID does not exist"
May 29 17:59:14.031576 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.031535    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.027147157 podStartE2EDuration="6.031521884s" podCreationTimestamp="2026-05-29 17:59:08 +0000 UTC" firstStartedPulling="2026-05-29 17:59:09.971566021 +0000 UTC m=+125.027359363" lastFinishedPulling="2026-05-29 17:59:12.975940749 +0000 UTC m=+128.031734090" observedRunningTime="2026-05-29 17:59:14.029528998 +0000 UTC m=+129.085322371" watchObservedRunningTime="2026-05-29 17:59:14.031521884 +0000 UTC m=+129.087315245"
May 29 17:59:14.044062 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.044037    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77c8d57b45-7zx6m"]
May 29 17:59:14.047528 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.047510    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77c8d57b45-7zx6m"]
May 29 17:59:14.088478 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.088450    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjbr4\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-kube-api-access-mjbr4\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 17:59:14.088608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.088485    2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aac1a9f-6772-4611-bc33-ac522d5de5d1-image-registry-private-configuration\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 17:59:14.088608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.088505    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-registry-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 17:59:14.088608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.088519    2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aac1a9f-6772-4611-bc33-ac522d5de5d1-bound-sa-token\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 17:59:14.088608 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:14.088531    2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aac1a9f-6772-4611-bc33-ac522d5de5d1-ca-trust-extracted\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 17:59:15.199879 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:15.199842    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:59:15.202166 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:15.202129    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19a8c87f-2157-4411-8e5a-b272ce1259f5-metrics-certs\") pod \"network-metrics-daemon-zsvdq\" (UID: \"19a8c87f-2157-4411-8e5a-b272ce1259f5\") " pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:59:15.276084 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:15.276057    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b7d95\""
May 29 17:59:15.283718 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:15.283699    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsvdq"
May 29 17:59:15.405215 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:15.405191    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zsvdq"]
May 29 17:59:15.407602 ip-10-0-133-167 kubenswrapper[2575]: W0529 17:59:15.407572    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a8c87f_2157_4411_8e5a_b272ce1259f5.slice/crio-930065418a3d75ca7e07710ef8062d05c3f7d1bb5010b90b5ffea6b149684c50 WatchSource:0}: Error finding container 930065418a3d75ca7e07710ef8062d05c3f7d1bb5010b90b5ffea6b149684c50: Status 404 returned error can't find the container with id 930065418a3d75ca7e07710ef8062d05c3f7d1bb5010b90b5ffea6b149684c50
May 29 17:59:15.468074 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:15.467986    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aac1a9f-6772-4611-bc33-ac522d5de5d1" path="/var/lib/kubelet/pods/4aac1a9f-6772-4611-bc33-ac522d5de5d1/volumes"
May 29 17:59:15.988316 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:15.988285    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-648d5c57bb-l9d6d"
May 29 17:59:16.008630 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:16.008590    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zsvdq" event={"ID":"19a8c87f-2157-4411-8e5a-b272ce1259f5","Type":"ContainerStarted","Data":"930065418a3d75ca7e07710ef8062d05c3f7d1bb5010b90b5ffea6b149684c50"}
May 29 17:59:17.013100 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:17.013065    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zsvdq" event={"ID":"19a8c87f-2157-4411-8e5a-b272ce1259f5","Type":"ContainerStarted","Data":"1fe890bb79b81ca05beacdb1feb6a881186a1cd31a73ee6182ac0c3172216ba4"}
May 29 17:59:17.013100 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:17.013100    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zsvdq" event={"ID":"19a8c87f-2157-4411-8e5a-b272ce1259f5","Type":"ContainerStarted","Data":"18db25977d4597946c25aa3ac46a9ae75678dcfd35e155984ec221018bfedcd2"}
May 29 17:59:17.040398 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:17.040336    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zsvdq" podStartSLOduration=131.117957292 podStartE2EDuration="2m12.040321941s" podCreationTimestamp="2026-05-29 17:57:05 +0000 UTC" firstStartedPulling="2026-05-29 17:59:15.409668903 +0000 UTC m=+130.465462257" lastFinishedPulling="2026-05-29 17:59:16.332033554 +0000 UTC m=+131.387826906" observedRunningTime="2026-05-29 17:59:17.03960504 +0000 UTC m=+132.095398407" watchObservedRunningTime="2026-05-29 17:59:17.040321941 +0000 UTC m=+132.096115302"
May 29 17:59:18.682839 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:18.682796    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 17:59:26.880360 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:26.880320    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:26.880360 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:26.880361    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:33.062731 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:33.062655    2575 generic.go:358] "Generic (PLEG): container finished" podID="3aee17dc-28c0-425e-9e7b-694f66cd6459" containerID="c05153fd31eef01c882b9dc90ea0ba765b87eb37c93ea929b3ed808eb227c63d" exitCode=0
May 29 17:59:33.063070 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:33.062729    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7" event={"ID":"3aee17dc-28c0-425e-9e7b-694f66cd6459","Type":"ContainerDied","Data":"c05153fd31eef01c882b9dc90ea0ba765b87eb37c93ea929b3ed808eb227c63d"}
May 29 17:59:33.063070 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:33.063014    2575 scope.go:117] "RemoveContainer" containerID="c05153fd31eef01c882b9dc90ea0ba765b87eb37c93ea929b3ed808eb227c63d"
May 29 17:59:34.067717 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:34.067685    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-78pw7" event={"ID":"3aee17dc-28c0-425e-9e7b-694f66cd6459","Type":"ContainerStarted","Data":"a2cdb76aaf833c43a0a8d573c0b4a4e6d0880dcb1ed934d1e702ec1d74c79265"}
May 29 17:59:46.885599 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:46.885571    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:46.889344 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:46.889327    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-746d756ffc-l9sj2"
May 29 17:59:48.110833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:48.110736    2575 generic.go:358] "Generic (PLEG): container finished" podID="5832977c-189f-4698-a7e6-790da79f5b5a" containerID="9fbf5298c025cf8d4d04ace19ba620f3f4e2ad5e768e8df629f51d7a02b6a8dd" exitCode=0
May 29 17:59:48.110833 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:48.110819    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs" event={"ID":"5832977c-189f-4698-a7e6-790da79f5b5a","Type":"ContainerDied","Data":"9fbf5298c025cf8d4d04ace19ba620f3f4e2ad5e768e8df629f51d7a02b6a8dd"}
May 29 17:59:48.111254 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:48.111147    2575 scope.go:117] "RemoveContainer" containerID="9fbf5298c025cf8d4d04ace19ba620f3f4e2ad5e768e8df629f51d7a02b6a8dd"
May 29 17:59:49.115908 ip-10-0-133-167 kubenswrapper[2575]: I0529 17:59:49.115835    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-l7tvs" event={"ID":"5832977c-189f-4698-a7e6-790da79f5b5a","Type":"ContainerStarted","Data":"d4d6ab102301bfba7d6125f0a3897d7cc2985b72dc847c0e830f89316bbe2ba0"}
May 29 18:00:08.682870 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:08.682835    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:08.698439 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:08.698414    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:09.190003 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:09.189976    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:22.493141 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:22.493099    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 18:00:22.493747 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:22.493612    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="alertmanager" containerID="cri-o://d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7" gracePeriod=120
May 29 18:00:22.493747 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:22.493660    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy" containerID="cri-o://b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8" gracePeriod=120
May 29 18:00:22.493747 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:22.493680    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy-web" containerID="cri-o://4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179" gracePeriod=120
May 29 18:00:22.493747 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:22.493621    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="prom-label-proxy" containerID="cri-o://79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b" gracePeriod=120
May 29 18:00:22.494046 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:22.493692    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy-metric" containerID="cri-o://6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20" gracePeriod=120
May 29 18:00:22.494046 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:22.493692    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="config-reloader" containerID="cri-o://c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0" gracePeriod=120
May 29 18:00:23.221969 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.221937    2575 generic.go:358] "Generic (PLEG): container finished" podID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerID="79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b" exitCode=0
May 29 18:00:23.221969 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.221961    2575 generic.go:358] "Generic (PLEG): container finished" podID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerID="b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8" exitCode=0
May 29 18:00:23.221969 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.221967    2575 generic.go:358] "Generic (PLEG): container finished" podID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerID="c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0" exitCode=0
May 29 18:00:23.221969 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.221973    2575 generic.go:358] "Generic (PLEG): container finished" podID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerID="d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7" exitCode=0
May 29 18:00:23.222256 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.222009    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerDied","Data":"79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b"}
May 29 18:00:23.222256 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.222039    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerDied","Data":"b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8"}
May 29 18:00:23.222256 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.222049    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerDied","Data":"c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0"}
May 29 18:00:23.222256 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.222057    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerDied","Data":"d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7"}
May 29 18:00:23.738955 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.738933    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:23.900629 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900600    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-volume\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.900791 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900646    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-cluster-tls-config\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.900791 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900675    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.900791 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900699    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-main-db\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.900791 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900724    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-tls-assets\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.900791 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900742    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-trusted-ca-bundle\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.900791 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900759    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.900791 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900777    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-web-config\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.901145 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900798    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2pz7\" (UniqueName: \"kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-kube-api-access-c2pz7\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.901145 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900832    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-out\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.901145 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900860    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-metrics-client-ca\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.901145 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900930    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-main-tls\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.901145 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.900997    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-web\") pod \"948ca724-ab0b-487d-875c-b8de8ba1a95b\" (UID: \"948ca724-ab0b-487d-875c-b8de8ba1a95b\") "
May 29 18:00:23.901145 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.901089    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:00:23.901481 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.901343    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-main-db\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:23.903439 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.903359    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:23.903827 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.903735    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:23.903827 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.903788    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:23.904179 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.904126    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-kube-api-access-c2pz7" (OuterVolumeSpecName: "kube-api-access-c2pz7") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "kube-api-access-c2pz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:00:23.904269 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.904215    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:00:23.904269 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.904217    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-volume" (OuterVolumeSpecName: "config-volume") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:23.904269 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.904236    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:23.904431 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.904312    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:23.905211 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.905185    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:23.905512 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.905485    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-out" (OuterVolumeSpecName: "config-out") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:00:23.908644 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.908572    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:23.915254 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:23.915231    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-web-config" (OuterVolumeSpecName: "web-config") pod "948ca724-ab0b-487d-875c-b8de8ba1a95b" (UID: "948ca724-ab0b-487d-875c-b8de8ba1a95b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:24.002739 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002689    2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-cluster-tls-config\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.002739 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002733    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.002739 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002747    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-tls-assets\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.002739 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002757    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.003006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002768    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.003006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002777    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-web-config\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.003006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002786    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2pz7\" (UniqueName: \"kubernetes.io/projected/948ca724-ab0b-487d-875c-b8de8ba1a95b-kube-api-access-c2pz7\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.003006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002794    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-out\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.003006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002802    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/948ca724-ab0b-487d-875c-b8de8ba1a95b-metrics-client-ca\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.003006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002811    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-main-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.003006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002819    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.003006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.002828    2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/948ca724-ab0b-487d-875c-b8de8ba1a95b-config-volume\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:24.228222 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.228117    2575 generic.go:358] "Generic (PLEG): container finished" podID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerID="6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20" exitCode=0
May 29 18:00:24.228222 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.228140    2575 generic.go:358] "Generic (PLEG): container finished" podID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerID="4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179" exitCode=0
May 29 18:00:24.228448 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.228209    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerDied","Data":"6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20"}
May 29 18:00:24.228448 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.228252    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerDied","Data":"4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179"}
May 29 18:00:24.228448 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.228264    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"948ca724-ab0b-487d-875c-b8de8ba1a95b","Type":"ContainerDied","Data":"d98c9d350f1e3eea2c3ba340062e432917528630283ac47c3f627cf074176dcd"}
May 29 18:00:24.228448 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.228268    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.228448 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.228278    2575 scope.go:117] "RemoveContainer" containerID="79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b"
May 29 18:00:24.235845 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.235823    2575 scope.go:117] "RemoveContainer" containerID="6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20"
May 29 18:00:24.243265 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.243247    2575 scope.go:117] "RemoveContainer" containerID="b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8"
May 29 18:00:24.249627 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.249611    2575 scope.go:117] "RemoveContainer" containerID="4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179"
May 29 18:00:24.255919 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.255895    2575 scope.go:117] "RemoveContainer" containerID="c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0"
May 29 18:00:24.259103 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.259079    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 18:00:24.260902 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.260882    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 18:00:24.264099 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.264082    2575 scope.go:117] "RemoveContainer" containerID="d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7"
May 29 18:00:24.270757 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.270740    2575 scope.go:117] "RemoveContainer" containerID="8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6"
May 29 18:00:24.277107 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.277093    2575 scope.go:117] "RemoveContainer" containerID="79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b"
May 29 18:00:24.277381 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:24.277364    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b\": container with ID starting with 79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b not found: ID does not exist" containerID="79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b"
May 29 18:00:24.277450 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.277390    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b"} err="failed to get container status \"79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b\": rpc error: code = NotFound desc = could not find container \"79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b\": container with ID starting with 79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b not found: ID does not exist"
May 29 18:00:24.277450 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.277411    2575 scope.go:117] "RemoveContainer" containerID="6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20"
May 29 18:00:24.277632 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:24.277615    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20\": container with ID starting with 6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20 not found: ID does not exist" containerID="6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20"
May 29 18:00:24.277680 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.277640    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20"} err="failed to get container status \"6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20\": rpc error: code = NotFound desc = could not find container \"6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20\": container with ID starting with 6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20 not found: ID does not exist"
May 29 18:00:24.277680 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.277656    2575 scope.go:117] "RemoveContainer" containerID="b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8"
May 29 18:00:24.277873 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:24.277858    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8\": container with ID starting with b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8 not found: ID does not exist" containerID="b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8"
May 29 18:00:24.277917 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.277876    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8"} err="failed to get container status \"b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8\": rpc error: code = NotFound desc = could not find container \"b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8\": container with ID starting with b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8 not found: ID does not exist"
May 29 18:00:24.277917 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.277898    2575 scope.go:117] "RemoveContainer" containerID="4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179"
May 29 18:00:24.278091 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:24.278073    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179\": container with ID starting with 4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179 not found: ID does not exist" containerID="4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179"
May 29 18:00:24.278270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.278099    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179"} err="failed to get container status \"4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179\": rpc error: code = NotFound desc = could not find container \"4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179\": container with ID starting with 4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179 not found: ID does not exist"
May 29 18:00:24.278270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.278120    2575 scope.go:117] "RemoveContainer" containerID="c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0"
May 29 18:00:24.278609 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:24.278591    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0\": container with ID starting with c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0 not found: ID does not exist" containerID="c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0"
May 29 18:00:24.278609 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.278611    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0"} err="failed to get container status \"c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0\": rpc error: code = NotFound desc = could not find container \"c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0\": container with ID starting with c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0 not found: ID does not exist"
May 29 18:00:24.278746 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.278624    2575 scope.go:117] "RemoveContainer" containerID="d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7"
May 29 18:00:24.278811 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:24.278797    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7\": container with ID starting with d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7 not found: ID does not exist" containerID="d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7"
May 29 18:00:24.278854 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.278814    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7"} err="failed to get container status \"d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7\": rpc error: code = NotFound desc = could not find container \"d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7\": container with ID starting with d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7 not found: ID does not exist"
May 29 18:00:24.278854 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.278827    2575 scope.go:117] "RemoveContainer" containerID="8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6"
May 29 18:00:24.279039 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:24.279023    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6\": container with ID starting with 8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6 not found: ID does not exist" containerID="8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6"
May 29 18:00:24.279099 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279047    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6"} err="failed to get container status \"8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6\": rpc error: code = NotFound desc = could not find container \"8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6\": container with ID starting with 8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6 not found: ID does not exist"
May 29 18:00:24.279099 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279066    2575 scope.go:117] "RemoveContainer" containerID="79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b"
May 29 18:00:24.279284 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279267    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b"} err="failed to get container status \"79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b\": rpc error: code = NotFound desc = could not find container \"79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b\": container with ID starting with 79ac0be050551e277b68cc33549cbb4008629dd1dc08a5fc88325e0a93945a3b not found: ID does not exist"
May 29 18:00:24.279331 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279284    2575 scope.go:117] "RemoveContainer" containerID="6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20"
May 29 18:00:24.279441 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279426    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20"} err="failed to get container status \"6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20\": rpc error: code = NotFound desc = could not find container \"6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20\": container with ID starting with 6b2b5bc72cf98c0970216bc710b9eb0036e71a6f512e69d216ebca8573ca2a20 not found: ID does not exist"
May 29 18:00:24.279484 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279440    2575 scope.go:117] "RemoveContainer" containerID="b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8"
May 29 18:00:24.279612 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279596    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8"} err="failed to get container status \"b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8\": rpc error: code = NotFound desc = could not find container \"b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8\": container with ID starting with b5011df00022500fa87f07ca88a664cc8a0436a11fba1f8fa746b078236295d8 not found: ID does not exist"
May 29 18:00:24.279653 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279612    2575 scope.go:117] "RemoveContainer" containerID="4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179"
May 29 18:00:24.279785 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279766    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179"} err="failed to get container status \"4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179\": rpc error: code = NotFound desc = could not find container \"4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179\": container with ID starting with 4f83adaf77fbc91cd50acd2e27b9d6e56bf1a864b37f41a37d9d939a925b9179 not found: ID does not exist"
May 29 18:00:24.279827 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279788    2575 scope.go:117] "RemoveContainer" containerID="c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0"
May 29 18:00:24.279978 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279953    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0"} err="failed to get container status \"c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0\": rpc error: code = NotFound desc = could not find container \"c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0\": container with ID starting with c4f0dbe4f6dee51995a9d13f6f3d93e611b57d6cb38ae1ff7c2b6b1866563fa0 not found: ID does not exist"
May 29 18:00:24.280026 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.279979    2575 scope.go:117] "RemoveContainer" containerID="d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7"
May 29 18:00:24.280200 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.280184    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7"} err="failed to get container status \"d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7\": rpc error: code = NotFound desc = could not find container \"d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7\": container with ID starting with d2ced9b0cfb79c95e51abe8c6ac7576aa30114354d0f9521fc765fed4e8318f7 not found: ID does not exist"
May 29 18:00:24.280200 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.280200    2575 scope.go:117] "RemoveContainer" containerID="8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6"
May 29 18:00:24.280331 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.280315    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6"} err="failed to get container status \"8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6\": rpc error: code = NotFound desc = could not find container \"8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6\": container with ID starting with 8c7577b0305938b1a95fd8cdeffb6cbbc29e13c8f99ede9135f83783ef019ec6 not found: ID does not exist"
May 29 18:00:24.303234 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303210    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 18:00:24.303536 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303517    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy-web"
May 29 18:00:24.303536 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303537    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy-web"
May 29 18:00:24.303536 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303554    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="config-reloader"
May 29 18:00:24.303536 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303562    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="config-reloader"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303571    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy-metric"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303580    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy-metric"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303591    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aac1a9f-6772-4611-bc33-ac522d5de5d1" containerName="registry"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303598    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aac1a9f-6772-4611-bc33-ac522d5de5d1" containerName="registry"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303609    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="alertmanager"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303617    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="alertmanager"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303627    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303635    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303658    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="prom-label-proxy"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303666    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="prom-label-proxy"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303692    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="init-config-reloader"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303701    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="init-config-reloader"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303801    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="config-reloader"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303814    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="4aac1a9f-6772-4611-bc33-ac522d5de5d1" containerName="registry"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303827    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy-web"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303839    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303850    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="kube-rbac-proxy-metric"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303861    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="alertmanager"
May 29 18:00:24.303887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.303871    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" containerName="prom-label-proxy"
May 29 18:00:24.309328 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.309310    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.312976 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.312781    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 29 18:00:24.313112 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.313010    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 29 18:00:24.313112 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.313068    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 29 18:00:24.313339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.313168    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 29 18:00:24.313339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.313246    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 29 18:00:24.313339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.313257    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 29 18:00:24.313339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.313316    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 29 18:00:24.313530 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.313386    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 29 18:00:24.313646 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.313630    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h6986\""
May 29 18:00:24.318049 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.318031    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 29 18:00:24.323112 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.323091    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 18:00:24.405696 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.405640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0536b842-caa2-4257-b24e-ce6964483d98-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.405696 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.405694    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0536b842-caa2-4257-b24e-ce6964483d98-config-out\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.405943 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.405728    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.405943 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.405767    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0536b842-caa2-4257-b24e-ce6964483d98-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.405943 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.405793    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.405943 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.405810    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.405943 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.405839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.405943 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.405898    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0536b842-caa2-4257-b24e-ce6964483d98-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.406256 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.405951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh5hp\" (UniqueName: \"kubernetes.io/projected/0536b842-caa2-4257-b24e-ce6964483d98-kube-api-access-hh5hp\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.406256 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.405993    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.406256 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.406031    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-web-config\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.406256 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.406068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-config-volume\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.406256 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.406091    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0536b842-caa2-4257-b24e-ce6964483d98-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507350 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507251    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507350 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507303    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507350 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507346    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507612 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507372    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0536b842-caa2-4257-b24e-ce6964483d98-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507612 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507403    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hh5hp\" (UniqueName: \"kubernetes.io/projected/0536b842-caa2-4257-b24e-ce6964483d98-kube-api-access-hh5hp\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507612 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507529    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507612 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507579    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-web-config\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507618    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-config-volume\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507642    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0536b842-caa2-4257-b24e-ce6964483d98-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507679    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0536b842-caa2-4257-b24e-ce6964483d98-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507718    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0536b842-caa2-4257-b24e-ce6964483d98-config-out\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507745    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.507811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507793    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0536b842-caa2-4257-b24e-ce6964483d98-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.508108 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.507823    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0536b842-caa2-4257-b24e-ce6964483d98-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.510677 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.510397    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-config-volume\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.510677 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.510431    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.510677 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.510451    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.510677 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.510597    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0536b842-caa2-4257-b24e-ce6964483d98-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.510677 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.510624    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.510677 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.510631    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.510942 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.510685    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-web-config\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.510942 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.510911    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0536b842-caa2-4257-b24e-ce6964483d98-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.511028 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.511012    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0536b842-caa2-4257-b24e-ce6964483d98-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.511493 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.511474    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0536b842-caa2-4257-b24e-ce6964483d98-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.512040 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.512022    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0536b842-caa2-4257-b24e-ce6964483d98-config-out\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.516009 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.515989    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh5hp\" (UniqueName: \"kubernetes.io/projected/0536b842-caa2-4257-b24e-ce6964483d98-kube-api-access-hh5hp\") pod \"alertmanager-main-0\" (UID: \"0536b842-caa2-4257-b24e-ce6964483d98\") " pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.620466 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.620424    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 29 18:00:24.755347 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:24.752699    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 29 18:00:24.862265 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:24.862230    2575 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0536b842_caa2_4257_b24e_ce6964483d98.slice/crio-conmon-c4bff8208bf7c997450ba8f08b5ea26c71f25d4c968823dd88454feb7599e772.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0536b842_caa2_4257_b24e_ce6964483d98.slice/crio-c4bff8208bf7c997450ba8f08b5ea26c71f25d4c968823dd88454feb7599e772.scope\": RecentStats: unable to find data in memory cache]"
May 29 18:00:25.231970 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:25.231937    2575 generic.go:358] "Generic (PLEG): container finished" podID="0536b842-caa2-4257-b24e-ce6964483d98" containerID="c4bff8208bf7c997450ba8f08b5ea26c71f25d4c968823dd88454feb7599e772" exitCode=0
May 29 18:00:25.232182 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:25.232021    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0536b842-caa2-4257-b24e-ce6964483d98","Type":"ContainerDied","Data":"c4bff8208bf7c997450ba8f08b5ea26c71f25d4c968823dd88454feb7599e772"}
May 29 18:00:25.232182 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:25.232056    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0536b842-caa2-4257-b24e-ce6964483d98","Type":"ContainerStarted","Data":"45fcbb85aaa54a3c716a12fbc4dde3b8573b16c698dea698be6cbb50f9e9e16c"}
May 29 18:00:25.468655 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:25.468625    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="948ca724-ab0b-487d-875c-b8de8ba1a95b" path="/var/lib/kubelet/pods/948ca724-ab0b-487d-875c-b8de8ba1a95b/volumes"
May 29 18:00:26.239365 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.239324    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0536b842-caa2-4257-b24e-ce6964483d98","Type":"ContainerStarted","Data":"d2be065c133963b4ed2cd3749f08aa414ab37d1df4f51fcf1527b47f80a5a0ad"}
May 29 18:00:26.239365 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.239369    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0536b842-caa2-4257-b24e-ce6964483d98","Type":"ContainerStarted","Data":"b7ea7331768308f6d81d00325549cd1cc569eeda5f35e309036c0ca7f334eac5"}
May 29 18:00:26.239802 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.239382    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0536b842-caa2-4257-b24e-ce6964483d98","Type":"ContainerStarted","Data":"326d2507ef7e7bbba475d4a847a2fdfe4d8a955e7434734d7baced87d16e0ea9"}
May 29 18:00:26.239802 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.239396    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0536b842-caa2-4257-b24e-ce6964483d98","Type":"ContainerStarted","Data":"89c96d9d1eb0cbae637b28303df2488541d9d1c3a24243a8a61756ee91219b38"}
May 29 18:00:26.239802 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.239407    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0536b842-caa2-4257-b24e-ce6964483d98","Type":"ContainerStarted","Data":"7c65129e23daee9c41af33cca36b527d149825547cefd0df830d7280ab148ee4"}
May 29 18:00:26.239802 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.239419    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0536b842-caa2-4257-b24e-ce6964483d98","Type":"ContainerStarted","Data":"231138caad163f0a74b9e5029dc9c86b432a0e9dcb7e585554357285c2f2c9ba"}
May 29 18:00:26.270018 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.269971    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.269952125 podStartE2EDuration="2.269952125s" podCreationTimestamp="2026-05-29 18:00:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:00:26.266976996 +0000 UTC m=+201.322770361" watchObservedRunningTime="2026-05-29 18:00:26.269952125 +0000 UTC m=+201.325745487"
May 29 18:00:26.745659 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.745619    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 18:00:26.746389 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.746150    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="prometheus" containerID="cri-o://999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512" gracePeriod=600
May 29 18:00:26.746389 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.746208    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="thanos-sidecar" containerID="cri-o://ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27" gracePeriod=600
May 29 18:00:26.746389 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.746196    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy" containerID="cri-o://06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb" gracePeriod=600
May 29 18:00:26.746389 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.746271    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy-web" containerID="cri-o://bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71" gracePeriod=600
May 29 18:00:26.746389 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.746298    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="config-reloader" containerID="cri-o://fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226" gracePeriod=600
May 29 18:00:26.746389 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:26.746316    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69" gracePeriod=600
May 29 18:00:27.100455 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.100426    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.132374 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132304    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-serving-certs-ca-bundle\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132374 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132364    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-kube-rbac-proxy\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132654 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132420    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-metrics-client-ca\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132654 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132444    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-tls-assets\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132654 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132466    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-db\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132654 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132498    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-trusted-ca-bundle\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132654 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132533    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-tls\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132654 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132552    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132654 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132567    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-kubelet-serving-ca-bundle\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132654 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132590    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config-out\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132654 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132614    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-web-config\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.132654 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132639    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.133513 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132666    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-rulefiles-0\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.133513 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132691    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-thanos-prometheus-http-client-file\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.133513 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132710    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-metrics-client-certs\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.133513 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132724    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-grpc-tls\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.133513 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132750    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.133513 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132789    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rm6c\" (UniqueName: \"kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-kube-api-access-7rm6c\") pod \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\" (UID: \"958a3e9c-5087-4e41-a4c1-e372c6de91ee\") "
May 29 18:00:27.133513 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.132823    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:27.133513 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.133073    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.135608 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.134435    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:27.135608 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.134446    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:27.135608 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.134499    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:00:27.137807 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.136833    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:27.137807 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.137535    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:00:27.137948 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.137843    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:27.138140 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.138115    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:27.138368 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.138339    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:27.138629 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.138590    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:00:27.138629 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.138613    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:27.138791 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.138643    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:27.138946 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.138904    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-kube-api-access-7rm6c" (OuterVolumeSpecName: "kube-api-access-7rm6c") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "kube-api-access-7rm6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:00:27.140701 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.140660    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config-out" (OuterVolumeSpecName: "config-out") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:00:27.141218 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.141181    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:27.141909 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.141879    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:27.142112 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.142090    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config" (OuterVolumeSpecName: "config") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:27.152299 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.152265    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-web-config" (OuterVolumeSpecName: "web-config") pod "958a3e9c-5087-4e41-a4c1-e372c6de91ee" (UID: "958a3e9c-5087-4e41-a4c1-e372c6de91ee"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:00:27.234480 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234450    2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-kube-rbac-proxy\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234480 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234473    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-metrics-client-ca\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234480 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234483    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-tls-assets\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234493    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-db\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234503    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234512    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234521    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234531    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234539    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config-out\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234548    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-web-config\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234556    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234565    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/958a3e9c-5087-4e41-a4c1-e372c6de91ee-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234574    2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234583    2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-metrics-client-certs\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234592    2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-secret-grpc-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234600    2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/958a3e9c-5087-4e41-a4c1-e372c6de91ee-config\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.234699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.234609    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rm6c\" (UniqueName: \"kubernetes.io/projected/958a3e9c-5087-4e41-a4c1-e372c6de91ee-kube-api-access-7rm6c\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:00:27.245917 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.245886    2575 generic.go:358] "Generic (PLEG): container finished" podID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerID="c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69" exitCode=0
May 29 18:00:27.245917 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.245910    2575 generic.go:358] "Generic (PLEG): container finished" podID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerID="06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb" exitCode=0
May 29 18:00:27.245917 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.245916    2575 generic.go:358] "Generic (PLEG): container finished" podID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerID="bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71" exitCode=0
May 29 18:00:27.245917 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.245922    2575 generic.go:358] "Generic (PLEG): container finished" podID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerID="ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27" exitCode=0
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.245927    2575 generic.go:358] "Generic (PLEG): container finished" podID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerID="fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226" exitCode=0
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.245932    2575 generic.go:358] "Generic (PLEG): container finished" podID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerID="999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512" exitCode=0
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.245978    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerDied","Data":"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"}
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.246014    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.246025    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerDied","Data":"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"}
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.246046    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerDied","Data":"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"}
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.246061    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerDied","Data":"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"}
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.246075    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerDied","Data":"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"}
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.246089    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerDied","Data":"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"}
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.246102    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"958a3e9c-5087-4e41-a4c1-e372c6de91ee","Type":"ContainerDied","Data":"55309edf2179ae5b88ed34ccbb8934e17a63a403a399444687212f46864ea245"}
May 29 18:00:27.246454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.246107    2575 scope.go:117] "RemoveContainer" containerID="c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"
May 29 18:00:27.253847 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.253737    2575 scope.go:117] "RemoveContainer" containerID="06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"
May 29 18:00:27.261016 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.260996    2575 scope.go:117] "RemoveContainer" containerID="bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"
May 29 18:00:27.267544 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.267528    2575 scope.go:117] "RemoveContainer" containerID="ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"
May 29 18:00:27.273575 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.273521    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 18:00:27.274589 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.274465    2575 scope.go:117] "RemoveContainer" containerID="fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"
May 29 18:00:27.277132 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.277108    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 18:00:27.281286 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.281261    2575 scope.go:117] "RemoveContainer" containerID="999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"
May 29 18:00:27.287936 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.287912    2575 scope.go:117] "RemoveContainer" containerID="fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"
May 29 18:00:27.294065 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.294047    2575 scope.go:117] "RemoveContainer" containerID="c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"
May 29 18:00:27.294337 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:27.294316    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": container with ID starting with c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69 not found: ID does not exist" containerID="c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"
May 29 18:00:27.294398 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.294345    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"} err="failed to get container status \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": rpc error: code = NotFound desc = could not find container \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": container with ID starting with c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69 not found: ID does not exist"
May 29 18:00:27.294398 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.294363    2575 scope.go:117] "RemoveContainer" containerID="06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"
May 29 18:00:27.294579 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:27.294566    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": container with ID starting with 06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb not found: ID does not exist" containerID="06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"
May 29 18:00:27.294621 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.294582    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"} err="failed to get container status \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": rpc error: code = NotFound desc = could not find container \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": container with ID starting with 06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb not found: ID does not exist"
May 29 18:00:27.294621 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.294593    2575 scope.go:117] "RemoveContainer" containerID="bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"
May 29 18:00:27.294909 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:27.294859    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": container with ID starting with bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71 not found: ID does not exist" containerID="bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"
May 29 18:00:27.295027 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.294932    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"} err="failed to get container status \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": rpc error: code = NotFound desc = could not find container \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": container with ID starting with bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71 not found: ID does not exist"
May 29 18:00:27.295027 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.294986    2575 scope.go:117] "RemoveContainer" containerID="ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"
May 29 18:00:27.295311 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:27.295289    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": container with ID starting with ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27 not found: ID does not exist" containerID="ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"
May 29 18:00:27.295402 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.295318    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"} err="failed to get container status \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": rpc error: code = NotFound desc = could not find container \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": container with ID starting with ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27 not found: ID does not exist"
May 29 18:00:27.295402 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.295342    2575 scope.go:117] "RemoveContainer" containerID="fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"
May 29 18:00:27.295715 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:27.295698    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": container with ID starting with fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226 not found: ID does not exist" containerID="fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"
May 29 18:00:27.295775 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.295720    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"} err="failed to get container status \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": rpc error: code = NotFound desc = could not find container \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": container with ID starting with fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226 not found: ID does not exist"
May 29 18:00:27.295775 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.295735    2575 scope.go:117] "RemoveContainer" containerID="999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"
May 29 18:00:27.295970 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:27.295956    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": container with ID starting with 999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512 not found: ID does not exist" containerID="999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"
May 29 18:00:27.296047 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.295973    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"} err="failed to get container status \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": rpc error: code = NotFound desc = could not find container \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": container with ID starting with 999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512 not found: ID does not exist"
May 29 18:00:27.296047 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.295985    2575 scope.go:117] "RemoveContainer" containerID="fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"
May 29 18:00:27.296231 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:00:27.296212    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": container with ID starting with fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144 not found: ID does not exist" containerID="fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"
May 29 18:00:27.296302 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.296235    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"} err="failed to get container status \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": rpc error: code = NotFound desc = could not find container \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": container with ID starting with fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144 not found: ID does not exist"
May 29 18:00:27.296302 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.296248    2575 scope.go:117] "RemoveContainer" containerID="c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"
May 29 18:00:27.296479 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.296460    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"} err="failed to get container status \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": rpc error: code = NotFound desc = could not find container \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": container with ID starting with c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69 not found: ID does not exist"
May 29 18:00:27.296527 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.296480    2575 scope.go:117] "RemoveContainer" containerID="06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"
May 29 18:00:27.296712 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.296696    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"} err="failed to get container status \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": rpc error: code = NotFound desc = could not find container \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": container with ID starting with 06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb not found: ID does not exist"
May 29 18:00:27.296758 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.296712    2575 scope.go:117] "RemoveContainer" containerID="bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"
May 29 18:00:27.296908 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.296892    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"} err="failed to get container status \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": rpc error: code = NotFound desc = could not find container \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": container with ID starting with bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71 not found: ID does not exist"
May 29 18:00:27.296952 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.296908    2575 scope.go:117] "RemoveContainer" containerID="ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"
May 29 18:00:27.297120 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.297102    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"} err="failed to get container status \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": rpc error: code = NotFound desc = could not find container \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": container with ID starting with ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27 not found: ID does not exist"
May 29 18:00:27.297231 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.297119    2575 scope.go:117] "RemoveContainer" containerID="fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"
May 29 18:00:27.297373 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.297358    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"} err="failed to get container status \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": rpc error: code = NotFound desc = could not find container \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": container with ID starting with fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226 not found: ID does not exist"
May 29 18:00:27.297418 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.297374    2575 scope.go:117] "RemoveContainer" containerID="999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"
May 29 18:00:27.297558 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.297540    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"} err="failed to get container status \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": rpc error: code = NotFound desc = could not find container \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": container with ID starting with 999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512 not found: ID does not exist"
May 29 18:00:27.297558 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.297558    2575 scope.go:117] "RemoveContainer" containerID="fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"
May 29 18:00:27.297757 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.297739    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"} err="failed to get container status \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": rpc error: code = NotFound desc = could not find container \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": container with ID starting with fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144 not found: ID does not exist"
May 29 18:00:27.297757 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.297757    2575 scope.go:117] "RemoveContainer" containerID="c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"
May 29 18:00:27.297939 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.297922    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"} err="failed to get container status \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": rpc error: code = NotFound desc = could not find container \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": container with ID starting with c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69 not found: ID does not exist"
May 29 18:00:27.297987 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.297940    2575 scope.go:117] "RemoveContainer" containerID="06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"
May 29 18:00:27.298187 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.298144    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"} err="failed to get container status \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": rpc error: code = NotFound desc = could not find container \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": container with ID starting with 06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb not found: ID does not exist"
May 29 18:00:27.298187 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.298187    2575 scope.go:117] "RemoveContainer" containerID="bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"
May 29 18:00:27.298388 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.298373    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"} err="failed to get container status \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": rpc error: code = NotFound desc = could not find container \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": container with ID starting with bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71 not found: ID does not exist"
May 29 18:00:27.298432 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.298388    2575 scope.go:117] "RemoveContainer" containerID="ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"
May 29 18:00:27.298563 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.298548    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"} err="failed to get container status \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": rpc error: code = NotFound desc = could not find container \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": container with ID starting with ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27 not found: ID does not exist"
May 29 18:00:27.298563 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.298562    2575 scope.go:117] "RemoveContainer" containerID="fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"
May 29 18:00:27.298760 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.298742    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"} err="failed to get container status \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": rpc error: code = NotFound desc = could not find container \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": container with ID starting with fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226 not found: ID does not exist"
May 29 18:00:27.298799 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.298760    2575 scope.go:117] "RemoveContainer" containerID="999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"
May 29 18:00:27.298982 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.298962    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"} err="failed to get container status \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": rpc error: code = NotFound desc = could not find container \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": container with ID starting with 999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512 not found: ID does not exist"
May 29 18:00:27.299024 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.298984    2575 scope.go:117] "RemoveContainer" containerID="fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"
May 29 18:00:27.299195 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.299178    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"} err="failed to get container status \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": rpc error: code = NotFound desc = could not find container \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": container with ID starting with fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144 not found: ID does not exist"
May 29 18:00:27.299249 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.299196    2575 scope.go:117] "RemoveContainer" containerID="c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"
May 29 18:00:27.299379 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.299363    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"} err="failed to get container status \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": rpc error: code = NotFound desc = could not find container \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": container with ID starting with c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69 not found: ID does not exist"
May 29 18:00:27.299430 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.299378    2575 scope.go:117] "RemoveContainer" containerID="06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"
May 29 18:00:27.299576 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.299549    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"} err="failed to get container status \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": rpc error: code = NotFound desc = could not find container \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": container with ID starting with 06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb not found: ID does not exist"
May 29 18:00:27.299576 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.299569    2575 scope.go:117] "RemoveContainer" containerID="bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"
May 29 18:00:27.299732 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.299717    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"} err="failed to get container status \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": rpc error: code = NotFound desc = could not find container \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": container with ID starting with bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71 not found: ID does not exist"
May 29 18:00:27.299732 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.299731    2575 scope.go:117] "RemoveContainer" containerID="ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"
May 29 18:00:27.299949 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.299923    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"} err="failed to get container status \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": rpc error: code = NotFound desc = could not find container \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": container with ID starting with ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27 not found: ID does not exist"
May 29 18:00:27.299949 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.299944    2575 scope.go:117] "RemoveContainer" containerID="fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"
May 29 18:00:27.300149 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.300132    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"} err="failed to get container status \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": rpc error: code = NotFound desc = could not find container \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": container with ID starting with fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226 not found: ID does not exist"
May 29 18:00:27.300218 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.300150    2575 scope.go:117] "RemoveContainer" containerID="999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"
May 29 18:00:27.300350 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.300333    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"} err="failed to get container status \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": rpc error: code = NotFound desc = could not find container \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": container with ID starting with 999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512 not found: ID does not exist"
May 29 18:00:27.300413 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.300350    2575 scope.go:117] "RemoveContainer" containerID="fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"
May 29 18:00:27.300532 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.300511    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"} err="failed to get container status \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": rpc error: code = NotFound desc = could not find container \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": container with ID starting with fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144 not found: ID does not exist"
May 29 18:00:27.300591 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.300533    2575 scope.go:117] "RemoveContainer" containerID="c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"
May 29 18:00:27.300735 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.300719    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"} err="failed to get container status \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": rpc error: code = NotFound desc = could not find container \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": container with ID starting with c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69 not found: ID does not exist"
May 29 18:00:27.300787 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.300735    2575 scope.go:117] "RemoveContainer" containerID="06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"
May 29 18:00:27.300929 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.300912    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"} err="failed to get container status \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": rpc error: code = NotFound desc = could not find container \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": container with ID starting with 06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb not found: ID does not exist"
May 29 18:00:27.300985 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.300929    2575 scope.go:117] "RemoveContainer" containerID="bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"
May 29 18:00:27.301098 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.301083    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"} err="failed to get container status \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": rpc error: code = NotFound desc = could not find container \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": container with ID starting with bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71 not found: ID does not exist"
May 29 18:00:27.301141 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.301098    2575 scope.go:117] "RemoveContainer" containerID="ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"
May 29 18:00:27.301361 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.301344    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"} err="failed to get container status \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": rpc error: code = NotFound desc = could not find container \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": container with ID starting with ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27 not found: ID does not exist"
May 29 18:00:27.301426 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.301360    2575 scope.go:117] "RemoveContainer" containerID="fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"
May 29 18:00:27.301562 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.301544    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"} err="failed to get container status \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": rpc error: code = NotFound desc = could not find container \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": container with ID starting with fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226 not found: ID does not exist"
May 29 18:00:27.301562 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.301563    2575 scope.go:117] "RemoveContainer" containerID="999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"
May 29 18:00:27.301746 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.301729    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"} err="failed to get container status \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": rpc error: code = NotFound desc = could not find container \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": container with ID starting with 999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512 not found: ID does not exist"
May 29 18:00:27.301785 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.301748    2575 scope.go:117] "RemoveContainer" containerID="fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"
May 29 18:00:27.301968 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.301951    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"} err="failed to get container status \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": rpc error: code = NotFound desc = could not find container \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": container with ID starting with fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144 not found: ID does not exist"
May 29 18:00:27.302018 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.301970    2575 scope.go:117] "RemoveContainer" containerID="c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"
May 29 18:00:27.302180 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.302147    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69"} err="failed to get container status \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": rpc error: code = NotFound desc = could not find container \"c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69\": container with ID starting with c26aa44daf3383c8f020a886bf474f5c3291d620dc0e7e59a45b2aeed6be5d69 not found: ID does not exist"
May 29 18:00:27.302180 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.302178    2575 scope.go:117] "RemoveContainer" containerID="06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"
May 29 18:00:27.302403 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.302382    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb"} err="failed to get container status \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": rpc error: code = NotFound desc = could not find container \"06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb\": container with ID starting with 06b102a19ed18f89babb8988249334bb7ccf11fa2ee6e36ab873af6d78cae7eb not found: ID does not exist"
May 29 18:00:27.302403 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.302398    2575 scope.go:117] "RemoveContainer" containerID="bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"
May 29 18:00:27.302597 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.302582    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71"} err="failed to get container status \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": rpc error: code = NotFound desc = could not find container \"bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71\": container with ID starting with bbc63fb4b89ce62dd1f02089bf1641648654132c33f86f3bc0c1481d51fb6e71 not found: ID does not exist"
May 29 18:00:27.302649 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.302596    2575 scope.go:117] "RemoveContainer" containerID="ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"
May 29 18:00:27.302787 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.302767    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27"} err="failed to get container status \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": rpc error: code = NotFound desc = could not find container \"ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27\": container with ID starting with ea8b9329664ca549f1969992556a8f253fcf98af5a9c8667d8028acc25998d27 not found: ID does not exist"
May 29 18:00:27.302825 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.302788    2575 scope.go:117] "RemoveContainer" containerID="fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"
May 29 18:00:27.302987 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.302965    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226"} err="failed to get container status \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": rpc error: code = NotFound desc = could not find container \"fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226\": container with ID starting with fa8cb749b568e01e14b816cb6a8f1ed2a130879ade7844286599616b8ebb0226 not found: ID does not exist"
May 29 18:00:27.303049 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.302988    2575 scope.go:117] "RemoveContainer" containerID="999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"
May 29 18:00:27.303182 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.303147    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512"} err="failed to get container status \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": rpc error: code = NotFound desc = could not find container \"999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512\": container with ID starting with 999f7812d6c0d008eb49eacb55e44b205390d778abf2776436226447cc03a512 not found: ID does not exist"
May 29 18:00:27.303259 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.303186    2575 scope.go:117] "RemoveContainer" containerID="fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"
May 29 18:00:27.303428 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.303406    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144"} err="failed to get container status \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": rpc error: code = NotFound desc = could not find container \"fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144\": container with ID starting with fd9f58d6b6bb5a0880725b9eef58f7eba61e201c9228eb5e619d46342ad60144 not found: ID does not exist"
May 29 18:00:27.307899 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.307876    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 18:00:27.308249 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308233    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy"
May 29 18:00:27.308249 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308250    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308264    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="config-reloader"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308269    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="config-reloader"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308276    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy-web"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308282    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy-web"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308290    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy-thanos"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308295    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy-thanos"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308303    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="init-config-reloader"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308308    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="init-config-reloader"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308314    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="prometheus"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308318    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="prometheus"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308328    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="thanos-sidecar"
May 29 18:00:27.308339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308333    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="thanos-sidecar"
May 29 18:00:27.308661 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308377    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy-thanos"
May 29 18:00:27.308661 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308387    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="thanos-sidecar"
May 29 18:00:27.308661 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308395    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy"
May 29 18:00:27.308661 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308401    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="prometheus"
May 29 18:00:27.308661 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308408    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="config-reloader"
May 29 18:00:27.308661 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.308414    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" containerName="kube-rbac-proxy-web"
May 29 18:00:27.314492 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.314473    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.317454 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317432    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 29 18:00:27.317561 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317510    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 29 18:00:27.317900 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317700    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 29 18:00:27.317900 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317706    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9sjkecoht317d\""
May 29 18:00:27.317900 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317771    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 29 18:00:27.317900 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317773    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 29 18:00:27.317900 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317774    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 29 18:00:27.317900 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317836    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 29 18:00:27.317900 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317866    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 29 18:00:27.317900 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317905    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vbdnz\""
May 29 18:00:27.318339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.317915    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 29 18:00:27.318339 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.318255    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 29 18:00:27.320888 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.320869    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 29 18:00:27.323212 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.323191    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 29 18:00:27.326508 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.326489    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 18:00:27.335746 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.335723    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-config\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.335832 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.335755    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.335832 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.335778    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lwsl\" (UniqueName: \"kubernetes.io/projected/7e8f4871-47c8-476a-a5fd-1155aad26670-kube-api-access-8lwsl\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.335832 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.335824    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e8f4871-47c8-476a-a5fd-1155aad26670-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.335993 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.335866    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.335993 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.335929    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.335993 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.335971    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e8f4871-47c8-476a-a5fd-1155aad26670-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336135 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336003    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336135 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336092    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336135 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336302 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336175    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336302 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336202    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336302 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336228    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336439 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336353    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336439 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336387    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336439 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336589 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336466    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e8f4871-47c8-476a-a5fd-1155aad26670-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.336589 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.336497    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437251 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437214    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-config\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437251 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437253    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437466 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437272    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwsl\" (UniqueName: \"kubernetes.io/projected/7e8f4871-47c8-476a-a5fd-1155aad26670-kube-api-access-8lwsl\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437466 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437298    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e8f4871-47c8-476a-a5fd-1155aad26670-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437466 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437394    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437633 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437470    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437633 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437514    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e8f4871-47c8-476a-a5fd-1155aad26670-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437633 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437543    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437633 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437608    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437823 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437640    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437823 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437669    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437823 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437699    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437823 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437727    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437823 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437786    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.437823 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437815    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.438100 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437849    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.438100 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437878    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e8f4871-47c8-476a-a5fd-1155aad26670-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.438100 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.437911    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.438283 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.438262    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.438384 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.438364    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.438995 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.438971    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.439140 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.438994    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e8f4871-47c8-476a-a5fd-1155aad26670-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.440302 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.440188    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.441040 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.440917    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.441114 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.441058    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e8f4871-47c8-476a-a5fd-1155aad26670-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.441766 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.441695    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-config\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.441766 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.441727    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e8f4871-47c8-476a-a5fd-1155aad26670-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.441986 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.441964    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.442174 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.442120    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.442174 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.442121    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.442403 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.442380    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.442515 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.442498    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.442838 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.442820    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e8f4871-47c8-476a-a5fd-1155aad26670-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.443383 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.443365    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.443603 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.443576    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e8f4871-47c8-476a-a5fd-1155aad26670-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.445670 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.445649    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lwsl\" (UniqueName: \"kubernetes.io/projected/7e8f4871-47c8-476a-a5fd-1155aad26670-kube-api-access-8lwsl\") pod \"prometheus-k8s-0\" (UID: \"7e8f4871-47c8-476a-a5fd-1155aad26670\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.468558 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.468534    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="958a3e9c-5087-4e41-a4c1-e372c6de91ee" path="/var/lib/kubelet/pods/958a3e9c-5087-4e41-a4c1-e372c6de91ee/volumes"
May 29 18:00:27.623845 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.623803    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:27.756666 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:27.756641    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 18:00:27.759091 ip-10-0-133-167 kubenswrapper[2575]: W0529 18:00:27.759066    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8f4871_47c8_476a_a5fd_1155aad26670.slice/crio-75eb499e90f1d4e28be43cfc0be29c77d2b5d7b2e33882e05ada4801dd6f7f75 WatchSource:0}: Error finding container 75eb499e90f1d4e28be43cfc0be29c77d2b5d7b2e33882e05ada4801dd6f7f75: Status 404 returned error can't find the container with id 75eb499e90f1d4e28be43cfc0be29c77d2b5d7b2e33882e05ada4801dd6f7f75
May 29 18:00:28.250217 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:28.250178    2575 generic.go:358] "Generic (PLEG): container finished" podID="7e8f4871-47c8-476a-a5fd-1155aad26670" containerID="2081d348c18184235a8ba96b60b9d9cf242ab54136df77c90ef9de0792966ef5" exitCode=0
May 29 18:00:28.250681 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:28.250218    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8f4871-47c8-476a-a5fd-1155aad26670","Type":"ContainerDied","Data":"2081d348c18184235a8ba96b60b9d9cf242ab54136df77c90ef9de0792966ef5"}
May 29 18:00:28.250681 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:28.250250    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8f4871-47c8-476a-a5fd-1155aad26670","Type":"ContainerStarted","Data":"75eb499e90f1d4e28be43cfc0be29c77d2b5d7b2e33882e05ada4801dd6f7f75"}
May 29 18:00:29.257417 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:29.257378    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8f4871-47c8-476a-a5fd-1155aad26670","Type":"ContainerStarted","Data":"f4c15f2b1a062834fb95c8f818f9bd3f1f3646b99656471d834222a8a5c76b7d"}
May 29 18:00:29.257792 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:29.257422    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8f4871-47c8-476a-a5fd-1155aad26670","Type":"ContainerStarted","Data":"060c1c48fd68752b23a221e861be156ed492a3a526625919f46de865b31dfd70"}
May 29 18:00:29.257792 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:29.257436    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8f4871-47c8-476a-a5fd-1155aad26670","Type":"ContainerStarted","Data":"60a410991fa305c3d32312a4ca84923c9026e9268fb6d548dfb243f26cd3504c"}
May 29 18:00:29.257792 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:29.257451    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8f4871-47c8-476a-a5fd-1155aad26670","Type":"ContainerStarted","Data":"a94a2321d56eed60b591da57431e03318d03c3cf6bfd768d1ab059ebf7511412"}
May 29 18:00:29.257792 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:29.257462    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8f4871-47c8-476a-a5fd-1155aad26670","Type":"ContainerStarted","Data":"2c2094b401d31da0478541feef0878960584c25d5c5a8e8eb2cc38f0a93edaac"}
May 29 18:00:29.257792 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:29.257474    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8f4871-47c8-476a-a5fd-1155aad26670","Type":"ContainerStarted","Data":"b27f88d7d31639c780dadc5f041b43c6d0fa6b967487e45be409bc0a74a6dbec"}
May 29 18:00:29.289350 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:29.289303    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.289287492 podStartE2EDuration="2.289287492s" podCreationTimestamp="2026-05-29 18:00:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:00:29.286520051 +0000 UTC m=+204.342313448" watchObservedRunningTime="2026-05-29 18:00:29.289287492 +0000 UTC m=+204.345080869"
May 29 18:00:32.624277 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:32.624230    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:00:59.415802 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.415765    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-bt5mq"]
May 29 18:00:59.419140 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.419124    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.422444 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.422415    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 29 18:00:59.430825 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.430800    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bt5mq"]
May 29 18:00:59.513924 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.513883    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a53c081c-9695-4f01-8518-64ab567e6496-original-pull-secret\") pod \"global-pull-secret-syncer-bt5mq\" (UID: \"a53c081c-9695-4f01-8518-64ab567e6496\") " pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.513924 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.513929    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a53c081c-9695-4f01-8518-64ab567e6496-dbus\") pod \"global-pull-secret-syncer-bt5mq\" (UID: \"a53c081c-9695-4f01-8518-64ab567e6496\") " pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.514130 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.513956    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a53c081c-9695-4f01-8518-64ab567e6496-kubelet-config\") pod \"global-pull-secret-syncer-bt5mq\" (UID: \"a53c081c-9695-4f01-8518-64ab567e6496\") " pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.615094 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.615061    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a53c081c-9695-4f01-8518-64ab567e6496-original-pull-secret\") pod \"global-pull-secret-syncer-bt5mq\" (UID: \"a53c081c-9695-4f01-8518-64ab567e6496\") " pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.615251 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.615107    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a53c081c-9695-4f01-8518-64ab567e6496-dbus\") pod \"global-pull-secret-syncer-bt5mq\" (UID: \"a53c081c-9695-4f01-8518-64ab567e6496\") " pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.615251 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.615146    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a53c081c-9695-4f01-8518-64ab567e6496-kubelet-config\") pod \"global-pull-secret-syncer-bt5mq\" (UID: \"a53c081c-9695-4f01-8518-64ab567e6496\") " pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.615348 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.615263    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a53c081c-9695-4f01-8518-64ab567e6496-kubelet-config\") pod \"global-pull-secret-syncer-bt5mq\" (UID: \"a53c081c-9695-4f01-8518-64ab567e6496\") " pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.615348 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.615311    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a53c081c-9695-4f01-8518-64ab567e6496-dbus\") pod \"global-pull-secret-syncer-bt5mq\" (UID: \"a53c081c-9695-4f01-8518-64ab567e6496\") " pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.617307 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.617285    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a53c081c-9695-4f01-8518-64ab567e6496-original-pull-secret\") pod \"global-pull-secret-syncer-bt5mq\" (UID: \"a53c081c-9695-4f01-8518-64ab567e6496\") " pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.728102 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.728019    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bt5mq"
May 29 18:00:59.856906 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:00:59.856853    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bt5mq"]
May 29 18:00:59.859272 ip-10-0-133-167 kubenswrapper[2575]: W0529 18:00:59.859244    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda53c081c_9695_4f01_8518_64ab567e6496.slice/crio-8f869ce3052c4a88538d0f83abe11816fcf0adf06464270133a11e16799e9889 WatchSource:0}: Error finding container 8f869ce3052c4a88538d0f83abe11816fcf0adf06464270133a11e16799e9889: Status 404 returned error can't find the container with id 8f869ce3052c4a88538d0f83abe11816fcf0adf06464270133a11e16799e9889
May 29 18:01:00.361823 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:01:00.361785    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bt5mq" event={"ID":"a53c081c-9695-4f01-8518-64ab567e6496","Type":"ContainerStarted","Data":"8f869ce3052c4a88538d0f83abe11816fcf0adf06464270133a11e16799e9889"}
May 29 18:01:04.376935 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:01:04.376899    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bt5mq" event={"ID":"a53c081c-9695-4f01-8518-64ab567e6496","Type":"ContainerStarted","Data":"2e5f926914d512d63b210f2a4fa5657a4d6d77ea141585ea0078b38770145d0b"}
May 29 18:01:04.394854 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:01:04.394808    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bt5mq" podStartSLOduration=1.1630772600000001 podStartE2EDuration="5.394795275s" podCreationTimestamp="2026-05-29 18:00:59 +0000 UTC" firstStartedPulling="2026-05-29 18:00:59.860987299 +0000 UTC m=+234.916780639" lastFinishedPulling="2026-05-29 18:01:04.092705311 +0000 UTC m=+239.148498654" observedRunningTime="2026-05-29 18:01:04.393059838 +0000 UTC m=+239.448853383" watchObservedRunningTime="2026-05-29 18:01:04.394795275 +0000 UTC m=+239.450588637"
May 29 18:01:27.624270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:01:27.624182    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:01:27.639461 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:01:27.639430    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:01:28.469562 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:01:28.469536    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 18:02:05.394386 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:02:05.394360    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:02:05.394910 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:02:05.394893    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:02:05.398980 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:02:05.398963    2575 kubelet.go:1628] "Image garbage collection succeeded"
May 29 18:07:05.417434 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:07:05.417408    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:07:05.418460 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:07:05.418441    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:09:25.302862 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.302830    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-c5knz"]
May 29 18:09:25.306304 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.306289    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-c5knz"
May 29 18:09:25.309312 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.309292    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 29 18:09:25.309312 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.309304    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 29 18:09:25.309479 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.309361    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 29 18:09:25.309479 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.309421    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-2k2zj\""
May 29 18:09:25.312939 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.312919    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-c5knz"]
May 29 18:09:25.459611 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.459573    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfjtm\" (UniqueName: \"kubernetes.io/projected/c4be4962-537e-44bd-94e3-ac8678c21d69-kube-api-access-wfjtm\") pod \"feast-test-apply-c5knz\" (UID: \"c4be4962-537e-44bd-94e3-ac8678c21d69\") " pod="test-ns-feast/feast-test-apply-c5knz"
May 29 18:09:25.560322 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.560214    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wfjtm\" (UniqueName: \"kubernetes.io/projected/c4be4962-537e-44bd-94e3-ac8678c21d69-kube-api-access-wfjtm\") pod \"feast-test-apply-c5knz\" (UID: \"c4be4962-537e-44bd-94e3-ac8678c21d69\") " pod="test-ns-feast/feast-test-apply-c5knz"
May 29 18:09:25.569941 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.569913    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfjtm\" (UniqueName: \"kubernetes.io/projected/c4be4962-537e-44bd-94e3-ac8678c21d69-kube-api-access-wfjtm\") pod \"feast-test-apply-c5knz\" (UID: \"c4be4962-537e-44bd-94e3-ac8678c21d69\") " pod="test-ns-feast/feast-test-apply-c5knz"
May 29 18:09:25.629688 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.629660    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-c5knz"
May 29 18:09:25.752309 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.752284    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-c5knz"]
May 29 18:09:25.755219 ip-10-0-133-167 kubenswrapper[2575]: W0529 18:09:25.755185    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4be4962_537e_44bd_94e3_ac8678c21d69.slice/crio-05a742e5b6b4a3bcc7b5df3da69becd89707ab8eefec8afde27bc3c66387de88 WatchSource:0}: Error finding container 05a742e5b6b4a3bcc7b5df3da69becd89707ab8eefec8afde27bc3c66387de88: Status 404 returned error can't find the container with id 05a742e5b6b4a3bcc7b5df3da69becd89707ab8eefec8afde27bc3c66387de88
May 29 18:09:25.759386 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.757801    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 18:09:25.849590 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:25.849504    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-c5knz" event={"ID":"c4be4962-537e-44bd-94e3-ac8678c21d69","Type":"ContainerStarted","Data":"05a742e5b6b4a3bcc7b5df3da69becd89707ab8eefec8afde27bc3c66387de88"}
May 29 18:09:31.873092 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:31.873056    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-c5knz" event={"ID":"c4be4962-537e-44bd-94e3-ac8678c21d69","Type":"ContainerStarted","Data":"2654dc19dfe983d03d03698b196fe79946eeef5b3e4318ba1f30d5edbfb1c47c"}
May 29 18:09:38.893427 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:38.893387    2575 generic.go:358] "Generic (PLEG): container finished" podID="c4be4962-537e-44bd-94e3-ac8678c21d69" containerID="2654dc19dfe983d03d03698b196fe79946eeef5b3e4318ba1f30d5edbfb1c47c" exitCode=0
May 29 18:09:38.893810 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:38.893463    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-c5knz" event={"ID":"c4be4962-537e-44bd-94e3-ac8678c21d69","Type":"ContainerDied","Data":"2654dc19dfe983d03d03698b196fe79946eeef5b3e4318ba1f30d5edbfb1c47c"}
May 29 18:09:39.897700 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:39.897664    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-c5knz" event={"ID":"c4be4962-537e-44bd-94e3-ac8678c21d69","Type":"ContainerStarted","Data":"68c9a1c20adcbdea9e81c6a69e789068e86abb6293acc11b6edba8a67aa675bd"}
May 29 18:09:39.915897 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:39.915845    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-c5knz" podStartSLOduration=9.459000628 podStartE2EDuration="14.915831432s" podCreationTimestamp="2026-05-29 18:09:25 +0000 UTC" firstStartedPulling="2026-05-29 18:09:25.757984447 +0000 UTC m=+740.813777790" lastFinishedPulling="2026-05-29 18:09:31.214815252 +0000 UTC m=+746.270608594" observedRunningTime="2026-05-29 18:09:39.914457874 +0000 UTC m=+754.970251236" watchObservedRunningTime="2026-05-29 18:09:39.915831432 +0000 UTC m=+754.971624826"
May 29 18:09:48.924232 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:48.924198    2575 generic.go:358] "Generic (PLEG): container finished" podID="c4be4962-537e-44bd-94e3-ac8678c21d69" containerID="68c9a1c20adcbdea9e81c6a69e789068e86abb6293acc11b6edba8a67aa675bd" exitCode=0
May 29 18:09:48.924595 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:48.924277    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-c5knz" event={"ID":"c4be4962-537e-44bd-94e3-ac8678c21d69","Type":"ContainerDied","Data":"68c9a1c20adcbdea9e81c6a69e789068e86abb6293acc11b6edba8a67aa675bd"}
May 29 18:09:50.059065 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:50.059038    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-c5knz"
May 29 18:09:50.163675 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:50.163645    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfjtm\" (UniqueName: \"kubernetes.io/projected/c4be4962-537e-44bd-94e3-ac8678c21d69-kube-api-access-wfjtm\") pod \"c4be4962-537e-44bd-94e3-ac8678c21d69\" (UID: \"c4be4962-537e-44bd-94e3-ac8678c21d69\") "
May 29 18:09:50.165758 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:50.165737    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4be4962-537e-44bd-94e3-ac8678c21d69-kube-api-access-wfjtm" (OuterVolumeSpecName: "kube-api-access-wfjtm") pod "c4be4962-537e-44bd-94e3-ac8678c21d69" (UID: "c4be4962-537e-44bd-94e3-ac8678c21d69"). InnerVolumeSpecName "kube-api-access-wfjtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:09:50.265130 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:50.265044    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfjtm\" (UniqueName: \"kubernetes.io/projected/c4be4962-537e-44bd-94e3-ac8678c21d69-kube-api-access-wfjtm\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:09:50.932569 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:50.932534    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-c5knz" event={"ID":"c4be4962-537e-44bd-94e3-ac8678c21d69","Type":"ContainerDied","Data":"05a742e5b6b4a3bcc7b5df3da69becd89707ab8eefec8afde27bc3c66387de88"}
May 29 18:09:50.932569 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:50.932567    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05a742e5b6b4a3bcc7b5df3da69becd89707ab8eefec8afde27bc3c66387de88"
May 29 18:09:50.932569 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:50.932550    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-c5knz"
May 29 18:09:51.120814 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:51.120769    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-c5knz_c4be4962-537e-44bd-94e3-ac8678c21d69/feast-0/0.log"
May 29 18:09:51.126098 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:09:51.126074    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-c5knz_c4be4962-537e-44bd-94e3-ac8678c21d69/feast-1/0.log"
May 29 18:10:24.953729 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:10:24.953647    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-c5knz"]
May 29 18:10:24.959551 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:10:24.959522    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-c5knz"]
May 29 18:10:25.468364 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:10:25.468331    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4be4962-537e-44bd-94e3-ac8678c21d69" path="/var/lib/kubelet/pods/c4be4962-537e-44bd-94e3-ac8678c21d69/volumes"
May 29 18:12:05.441047 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:12:05.441017    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:12:05.442861 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:12:05.442831    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:15:13.092243 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.092197    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"]
May 29 18:15:13.092683 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.092570    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4be4962-537e-44bd-94e3-ac8678c21d69" containerName="feast-1"
May 29 18:15:13.092683 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.092585    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4be4962-537e-44bd-94e3-ac8678c21d69" containerName="feast-1"
May 29 18:15:13.092683 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.092599    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4be4962-537e-44bd-94e3-ac8678c21d69" containerName="feast-0"
May 29 18:15:13.092683 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.092605    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4be4962-537e-44bd-94e3-ac8678c21d69" containerName="feast-0"
May 29 18:15:13.092683 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.092664    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="c4be4962-537e-44bd-94e3-ac8678c21d69" containerName="feast-1"
May 29 18:15:13.100258 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.100229    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.103246 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.103225    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 29 18:15:13.103246 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.103246    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 29 18:15:13.103448 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.103224    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 29 18:15:13.103513 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.103499    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 29 18:15:13.103574 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.103559    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 29 18:15:13.103618 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.103591    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-lcwvt\""
May 29 18:15:13.103618 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.103606    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 29 18:15:13.107230 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.107209    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"]
May 29 18:15:13.193917 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.193885    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.194094 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.193933    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.194094 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.193987    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22235ac3-26a6-40af-bbb3-b2feb240b897-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.194094 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.194020    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrqhb\" (UniqueName: \"kubernetes.io/projected/22235ac3-26a6-40af-bbb3-b2feb240b897-kube-api-access-nrqhb\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.194094 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.194051    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.194094 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.194089    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.295089 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.295050    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.295296 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.295108    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22235ac3-26a6-40af-bbb3-b2feb240b897-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.295296 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.295142    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nrqhb\" (UniqueName: \"kubernetes.io/projected/22235ac3-26a6-40af-bbb3-b2feb240b897-kube-api-access-nrqhb\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.295296 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.295186    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.295296 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:15:13.295206    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 29 18:15:13.295296 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:15:13.295283    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-online-tls podName:22235ac3-26a6-40af-bbb3-b2feb240b897 nodeName:}" failed. No retries permitted until 2026-05-29 18:15:13.795266841 +0000 UTC m=+1088.851060181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-online-tls") pod "feast-simple-feast-setup-664fcfcc58-4k28w" (UID: "22235ac3-26a6-40af-bbb3-b2feb240b897") : secret "feast-simple-feast-setup-online-tls" not found
May 29 18:15:13.295525 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.295216    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.295525 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.295362    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.295591 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.295551    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22235ac3-26a6-40af-bbb3-b2feb240b897-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.297703 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.297674    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.297703 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.297695    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.297842 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.297770    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.303960 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.303938    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrqhb\" (UniqueName: \"kubernetes.io/projected/22235ac3-26a6-40af-bbb3-b2feb240b897-kube-api-access-nrqhb\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.800103 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.800064    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:13.802500 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:13.802467    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-4k28w\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:14.031204 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:14.031148    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:14.159911 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:14.159886    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"]
May 29 18:15:14.160239 ip-10-0-133-167 kubenswrapper[2575]: W0529 18:15:14.160064    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22235ac3_26a6_40af_bbb3_b2feb240b897.slice/crio-2f9764a1fa760aa4d72b0c14e23a92d5ab2492add4dde4888800fdec6dc50c48 WatchSource:0}: Error finding container 2f9764a1fa760aa4d72b0c14e23a92d5ab2492add4dde4888800fdec6dc50c48: Status 404 returned error can't find the container with id 2f9764a1fa760aa4d72b0c14e23a92d5ab2492add4dde4888800fdec6dc50c48
May 29 18:15:14.161708 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:14.161692    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 18:15:14.914428 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:14.914389    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerStarted","Data":"2f9764a1fa760aa4d72b0c14e23a92d5ab2492add4dde4888800fdec6dc50c48"}
May 29 18:15:30.974073 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:30.974039    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerStarted","Data":"438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2"}
May 29 18:15:34.987556 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:34.987519    2575 generic.go:358] "Generic (PLEG): container finished" podID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerID="438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2" exitCode=0
May 29 18:15:34.987943 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:34.987592    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerDied","Data":"438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2"}
May 29 18:15:35.992118 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:35.992083    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerStarted","Data":"07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11"}
May 29 18:15:41.011821 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:41.011785    2575 generic.go:358] "Generic (PLEG): container finished" podID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerID="07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11" exitCode=0
May 29 18:15:41.012221 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:41.011856    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerDied","Data":"07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11"}
May 29 18:15:42.019616 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:42.019575    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerStarted","Data":"ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6"}
May 29 18:15:42.019616 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:42.019617    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerStarted","Data":"54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d"}
May 29 18:15:42.020225 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:42.019632    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerStarted","Data":"15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7"}
May 29 18:15:42.020225 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:42.019645    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerStarted","Data":"4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1"}
May 29 18:15:42.045552 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:42.045489    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podStartSLOduration=13.246377601 podStartE2EDuration="29.045471526s" podCreationTimestamp="2026-05-29 18:15:13 +0000 UTC" firstStartedPulling="2026-05-29 18:15:14.161816033 +0000 UTC m=+1089.217609372" lastFinishedPulling="2026-05-29 18:15:29.960909955 +0000 UTC m=+1105.016703297" observedRunningTime="2026-05-29 18:15:42.043254362 +0000 UTC m=+1117.099047748" watchObservedRunningTime="2026-05-29 18:15:42.045471526 +0000 UTC m=+1117.101264888"
May 29 18:15:44.032128 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.032096    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:44.032739 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.032712    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:44.032739 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.032740    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:44.032924 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.032753    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:44.032924 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.032765    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:44.032924 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.032777    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:44.032924 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.032788    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:44.032924 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.032800    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:44.034362 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.034187    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 29 18:15:44.034362 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.034226    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 29 18:15:44.034362 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.034266    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused"
May 29 18:15:44.034362 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:44.034227    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused"
May 29 18:15:47.032469 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:47.032436    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:47.032903 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:47.032573    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:47.032903 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:47.032807    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:47.033556 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:47.033529    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:47.033669 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:47.033625    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:47.033756 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:47.033739    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:47.036986 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:47.036959    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:15:47.041258 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:15:47.041236    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:16:05.471131 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:05.471101    2575 scope.go:117] "RemoveContainer" containerID="2654dc19dfe983d03d03698b196fe79946eeef5b3e4318ba1f30d5edbfb1c47c"
May 29 18:16:05.478265 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:05.478243    2575 scope.go:117] "RemoveContainer" containerID="68c9a1c20adcbdea9e81c6a69e789068e86abb6293acc11b6edba8a67aa675bd"
May 29 18:16:48.689687 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:48.689597    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"]
May 29 18:16:48.690291 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:48.689974    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="registry" containerID="cri-o://4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1" gracePeriod=30
May 29 18:16:48.690291 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:48.690017    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="offline" containerID="cri-o://54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d" gracePeriod=30
May 29 18:16:48.690291 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:48.690043    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="online" containerID="cri-o://15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7" gracePeriod=30
May 29 18:16:48.690291 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:48.690043    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="ui" containerID="cri-o://ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6" gracePeriod=30
May 29 18:16:49.096719 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.096632    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"]
May 29 18:16:49.100967 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.100942    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.103701 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.103680    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-mw9zm\""
May 29 18:16:49.113052 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.113025    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"]
May 29 18:16:49.159817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.159792    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.159964 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.159825    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.159964 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.159845    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.159964 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.159910    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6knf\" (UniqueName: \"kubernetes.io/projected/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-kube-api-access-w6knf\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.159964 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.159937    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.160147 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.159989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.233468 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.233437    2575 generic.go:358] "Generic (PLEG): container finished" podID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerID="ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6" exitCode=0
May 29 18:16:49.233629 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.233514    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerDied","Data":"ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6"}
May 29 18:16:49.260955 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.260917    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.261187 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.260981    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.261187 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.261004    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.261187 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.261023    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.261187 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.261057    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w6knf\" (UniqueName: \"kubernetes.io/projected/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-kube-api-access-w6knf\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.261427 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.261219    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.261487 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.261452    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.263633 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.263574    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.263758 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.263639    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.263944 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.263925    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.264003 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.263941    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.269346 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.269327    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6knf\" (UniqueName: \"kubernetes.io/projected/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-kube-api-access-w6knf\") pod \"feast-simple-feast-setup-664fcfcc58-l2hbv\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.413134 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.413108    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:16:49.546442 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:49.546414    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"]
May 29 18:16:49.548857 ip-10-0-133-167 kubenswrapper[2575]: W0529 18:16:49.548827    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c9747d6_ae55_48e1_9ead_c8d0c12c0883.slice/crio-2c73b476a0f5d362a7ccdb4575160e56a29b66111470bbe09e09b3551fda9269 WatchSource:0}: Error finding container 2c73b476a0f5d362a7ccdb4575160e56a29b66111470bbe09e09b3551fda9269: Status 404 returned error can't find the container with id 2c73b476a0f5d362a7ccdb4575160e56a29b66111470bbe09e09b3551fda9269
May 29 18:16:50.238978 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:50.238947    2575 generic.go:358] "Generic (PLEG): container finished" podID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerID="15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7" exitCode=0
May 29 18:16:50.239420 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:50.239023    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerDied","Data":"15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7"}
May 29 18:16:50.240169 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:50.240129    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerStarted","Data":"44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7"}
May 29 18:16:50.240269 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:50.240176    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerStarted","Data":"2c73b476a0f5d362a7ccdb4575160e56a29b66111470bbe09e09b3551fda9269"}
May 29 18:16:54.252923 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:54.252884    2575 generic.go:358] "Generic (PLEG): container finished" podID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerID="44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7" exitCode=0
May 29 18:16:54.253317 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:54.252956    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerDied","Data":"44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7"}
May 29 18:16:55.258089 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:55.258057    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerStarted","Data":"4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15"}
May 29 18:16:57.034245 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:57.034202    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 29 18:16:57.039235 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:16:57.039207    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 29 18:17:00.275040 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:00.275010    2575 generic.go:358] "Generic (PLEG): container finished" podID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerID="4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15" exitCode=0
May 29 18:17:00.275443 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:00.275087    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerDied","Data":"4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15"}
May 29 18:17:01.283371 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.283325    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerStarted","Data":"897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c"}
May 29 18:17:01.283371 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.283377    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerStarted","Data":"a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4"}
May 29 18:17:01.283919 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.283392    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerStarted","Data":"49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce"}
May 29 18:17:01.283919 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.283406    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerStarted","Data":"84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0"}
May 29 18:17:01.310306 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.310245    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podStartSLOduration=12.310226384 podStartE2EDuration="12.310226384s" podCreationTimestamp="2026-05-29 18:16:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:17:01.308464916 +0000 UTC m=+1196.364258279" watchObservedRunningTime="2026-05-29 18:17:01.310226384 +0000 UTC m=+1196.366019747"
May 29 18:17:01.413672 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.413554    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:01.413672 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.413605    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:01.413672 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.413619    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:01.413672 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.413631    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:01.415478 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.415433    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused"
May 29 18:17:01.415602 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.415487    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 29 18:17:01.415602 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.415498    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 29 18:17:01.415602 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:01.415448    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused"
May 29 18:17:04.414024 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:04.413887    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused"
May 29 18:17:04.414024 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:04.413911    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused"
May 29 18:17:04.414024 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:04.413920    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 29 18:17:04.414663 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:04.414374    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 29 18:17:05.485691 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:05.485656    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:17:05.486931 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:05.486908    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:17:07.033561 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.033513    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 29 18:17:07.039896 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.039861    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 29 18:17:07.414973 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.414938    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:07.415177 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.415089    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:07.415334 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.415315    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:07.415745 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.415638    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:07.415745 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.415665    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:07.415745 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.415678    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:07.415940 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.415882    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:07.415940 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.415936    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:07.416317 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.416294    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:07.419675 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:07.419657    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:08.313558 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:08.313530    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:08.316383 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:08.316368    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:17:17.033815 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:17.033771    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 29 18:17:17.034264 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:17.033903    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:17:17.038927 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:17.038902    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 29 18:17:17.039045 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:17.038980    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:17:19.333033 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.333013    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:17:19.349899 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.349871    2575 generic.go:358] "Generic (PLEG): container finished" podID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerID="54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d" exitCode=137
May 29 18:17:19.349899 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.349897    2575 generic.go:358] "Generic (PLEG): container finished" podID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerID="4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1" exitCode=137
May 29 18:17:19.350107 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.349938    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"
May 29 18:17:19.350107 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.349955    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerDied","Data":"54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d"}
May 29 18:17:19.350107 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.350003    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerDied","Data":"4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1"}
May 29 18:17:19.350107 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.350019    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w" event={"ID":"22235ac3-26a6-40af-bbb3-b2feb240b897","Type":"ContainerDied","Data":"2f9764a1fa760aa4d72b0c14e23a92d5ab2492add4dde4888800fdec6dc50c48"}
May 29 18:17:19.350107 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.350037    2575 scope.go:117] "RemoveContainer" containerID="ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6"
May 29 18:17:19.358536 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.358513    2575 scope.go:117] "RemoveContainer" containerID="54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d"
May 29 18:17:19.367288 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.367272    2575 scope.go:117] "RemoveContainer" containerID="15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7"
May 29 18:17:19.374972 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.374958    2575 scope.go:117] "RemoveContainer" containerID="4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1"
May 29 18:17:19.383568 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.383550    2575 scope.go:117] "RemoveContainer" containerID="07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11"
May 29 18:17:19.390466 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.390448    2575 scope.go:117] "RemoveContainer" containerID="438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2"
May 29 18:17:19.397032 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.397016    2575 scope.go:117] "RemoveContainer" containerID="ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6"
May 29 18:17:19.397301 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:17:19.397281    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6\": container with ID starting with ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6 not found: ID does not exist" containerID="ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6"
May 29 18:17:19.397386 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.397313    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6"} err="failed to get container status \"ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6\": rpc error: code = NotFound desc = could not find container \"ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6\": container with ID starting with ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6 not found: ID does not exist"
May 29 18:17:19.397386 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.397337    2575 scope.go:117] "RemoveContainer" containerID="54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d"
May 29 18:17:19.397574 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:17:19.397555    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d\": container with ID starting with 54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d not found: ID does not exist" containerID="54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d"
May 29 18:17:19.397614 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.397581    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d"} err="failed to get container status \"54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d\": rpc error: code = NotFound desc = could not find container \"54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d\": container with ID starting with 54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d not found: ID does not exist"
May 29 18:17:19.397614 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.397598    2575 scope.go:117] "RemoveContainer" containerID="15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7"
May 29 18:17:19.397808 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:17:19.397789    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7\": container with ID starting with 15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7 not found: ID does not exist" containerID="15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7"
May 29 18:17:19.397870 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.397816    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7"} err="failed to get container status \"15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7\": rpc error: code = NotFound desc = could not find container \"15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7\": container with ID starting with 15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7 not found: ID does not exist"
May 29 18:17:19.397870 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.397837    2575 scope.go:117] "RemoveContainer" containerID="4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1"
May 29 18:17:19.398031 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:17:19.398016    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1\": container with ID starting with 4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1 not found: ID does not exist" containerID="4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1"
May 29 18:17:19.398067 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.398036    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1"} err="failed to get container status \"4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1\": rpc error: code = NotFound desc = could not find container \"4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1\": container with ID starting with 4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1 not found: ID does not exist"
May 29 18:17:19.398067 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.398050    2575 scope.go:117] "RemoveContainer" containerID="07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11"
May 29 18:17:19.398252 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:17:19.398235    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11\": container with ID starting with 07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11 not found: ID does not exist" containerID="07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11"
May 29 18:17:19.398300 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.398257    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11"} err="failed to get container status \"07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11\": rpc error: code = NotFound desc = could not find container \"07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11\": container with ID starting with 07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11 not found: ID does not exist"
May 29 18:17:19.398300 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.398272    2575 scope.go:117] "RemoveContainer" containerID="438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2"
May 29 18:17:19.398487 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:17:19.398469    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2\": container with ID starting with 438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2 not found: ID does not exist" containerID="438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2"
May 29 18:17:19.398578 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.398486    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2"} err="failed to get container status \"438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2\": rpc error: code = NotFound desc = could not find container \"438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2\": container with ID starting with 438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2 not found: ID does not exist"
May 29 18:17:19.398578 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.398506    2575 scope.go:117] "RemoveContainer" containerID="ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6"
May 29 18:17:19.398728 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.398711    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6"} err="failed to get container status \"ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6\": rpc error: code = NotFound desc = could not find container \"ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6\": container with ID starting with ab7dcd89bc9b6011a0ff61ccf48febe7d7c1787209bfeafc7c03028d076a08c6 not found: ID does not exist"
May 29 18:17:19.398773 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.398731    2575 scope.go:117] "RemoveContainer" containerID="54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d"
May 29 18:17:19.398946 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.398927    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d"} err="failed to get container status \"54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d\": rpc error: code = NotFound desc = could not find container \"54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d\": container with ID starting with 54bc70cedba1aca30bae2a3173ccade4b5ce190f51dbd69d251755bc68d88e5d not found: ID does not exist"
May 29 18:17:19.399011 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.398948    2575 scope.go:117] "RemoveContainer" containerID="15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7"
May 29 18:17:19.399173 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.399134    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7"} err="failed to get container status \"15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7\": rpc error: code = NotFound desc = could not find container \"15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7\": container with ID starting with 15c7ff444ddd826eb6a749c14d32e1e9a719d7f46dcaf1e87ea9fed4ef4a7ff7 not found: ID does not exist"
May 29 18:17:19.399233 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.399178    2575 scope.go:117] "RemoveContainer" containerID="4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1"
May 29 18:17:19.399401 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.399384    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1"} err="failed to get container status \"4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1\": rpc error: code = NotFound desc = could not find container \"4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1\": container with ID starting with 4fa4b99f2c6a879c68c4abdc11129573f2563148ce45d63548f2e3d539aa48d1 not found: ID does not exist"
May 29 18:17:19.399458 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.399401    2575 scope.go:117] "RemoveContainer" containerID="07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11"
May 29 18:17:19.399600 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.399581    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11"} err="failed to get container status \"07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11\": rpc error: code = NotFound desc = could not find container \"07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11\": container with ID starting with 07ab56ab21fd2625ca3c974e9251e0ec5a6ff389e3589b33abaf5c3f26156a11 not found: ID does not exist"
May 29 18:17:19.399650 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.399602    2575 scope.go:117] "RemoveContainer" containerID="438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2"
May 29 18:17:19.399767 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.399752    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2"} err="failed to get container status \"438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2\": rpc error: code = NotFound desc = could not find container \"438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2\": container with ID starting with 438804227e1ff4111669db5f179d617c1c6357add07be798caad1cb6896f51d2 not found: ID does not exist"
May 29 18:17:19.445755 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.445732    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-registry-tls\") pod \"22235ac3-26a6-40af-bbb3-b2feb240b897\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") "
May 29 18:17:19.445906 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.445772    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-ui-tls\") pod \"22235ac3-26a6-40af-bbb3-b2feb240b897\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") "
May 29 18:17:19.445906 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.445801    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-offline-tls\") pod \"22235ac3-26a6-40af-bbb3-b2feb240b897\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") "
May 29 18:17:19.446087 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.445921    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-online-tls\") pod \"22235ac3-26a6-40af-bbb3-b2feb240b897\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") "
May 29 18:17:19.446087 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.446062    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22235ac3-26a6-40af-bbb3-b2feb240b897-feast-data\") pod \"22235ac3-26a6-40af-bbb3-b2feb240b897\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") "
May 29 18:17:19.446234 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.446126    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrqhb\" (UniqueName: \"kubernetes.io/projected/22235ac3-26a6-40af-bbb3-b2feb240b897-kube-api-access-nrqhb\") pod \"22235ac3-26a6-40af-bbb3-b2feb240b897\" (UID: \"22235ac3-26a6-40af-bbb3-b2feb240b897\") "
May 29 18:17:19.446660 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.446617    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22235ac3-26a6-40af-bbb3-b2feb240b897-feast-data" (OuterVolumeSpecName: "feast-data") pod "22235ac3-26a6-40af-bbb3-b2feb240b897" (UID: "22235ac3-26a6-40af-bbb3-b2feb240b897"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:17:19.448127 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.448104    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "22235ac3-26a6-40af-bbb3-b2feb240b897" (UID: "22235ac3-26a6-40af-bbb3-b2feb240b897"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:17:19.448238 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.448127    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "22235ac3-26a6-40af-bbb3-b2feb240b897" (UID: "22235ac3-26a6-40af-bbb3-b2feb240b897"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:17:19.448298 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.448274    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "22235ac3-26a6-40af-bbb3-b2feb240b897" (UID: "22235ac3-26a6-40af-bbb3-b2feb240b897"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:17:19.448341 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.448300    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-online-tls" (OuterVolumeSpecName: "online-tls") pod "22235ac3-26a6-40af-bbb3-b2feb240b897" (UID: "22235ac3-26a6-40af-bbb3-b2feb240b897"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:17:19.448377 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.448358    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22235ac3-26a6-40af-bbb3-b2feb240b897-kube-api-access-nrqhb" (OuterVolumeSpecName: "kube-api-access-nrqhb") pod "22235ac3-26a6-40af-bbb3-b2feb240b897" (UID: "22235ac3-26a6-40af-bbb3-b2feb240b897"). InnerVolumeSpecName "kube-api-access-nrqhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:17:19.546970 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.546936    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-registry-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:17:19.546970 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.546963    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-ui-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:17:19.546970 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.546972    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-offline-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:17:19.546970 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.546980    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22235ac3-26a6-40af-bbb3-b2feb240b897-online-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:17:19.547310 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.546989    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22235ac3-26a6-40af-bbb3-b2feb240b897-feast-data\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:17:19.547310 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.546999    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrqhb\" (UniqueName: \"kubernetes.io/projected/22235ac3-26a6-40af-bbb3-b2feb240b897-kube-api-access-nrqhb\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:17:19.669584 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.669545    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"]
May 29 18:17:19.673826 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:19.673803    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-4k28w"]
May 29 18:17:21.468222 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:17:21.468189    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" path="/var/lib/kubelet/pods/22235ac3-26a6-40af-bbb3-b2feb240b897/volumes"
May 29 18:18:10.877194 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877146    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"]
May 29 18:18:10.877626 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877617    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="ui"
May 29 18:18:10.877669 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877631    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="ui"
May 29 18:18:10.877669 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877643    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="feast-init"
May 29 18:18:10.877669 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877648    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="feast-init"
May 29 18:18:10.877669 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877656    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="offline"
May 29 18:18:10.877669 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877662    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="offline"
May 29 18:18:10.877817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877674    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="feast-apply"
May 29 18:18:10.877817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877679    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="feast-apply"
May 29 18:18:10.877817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877686    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="registry"
May 29 18:18:10.877817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877690    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="registry"
May 29 18:18:10.877817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877700    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="online"
May 29 18:18:10.877817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877705    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="online"
May 29 18:18:10.877817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877772    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="offline"
May 29 18:18:10.877817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877783    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="online"
May 29 18:18:10.877817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877789    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="ui"
May 29 18:18:10.877817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.877796    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="22235ac3-26a6-40af-bbb3-b2feb240b897" containerName="registry"
May 29 18:18:10.882564 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.882545    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:10.885518 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.885500    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 29 18:18:10.886732 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.886707    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 29 18:18:10.886842 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.886715    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 29 18:18:10.886842 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.886755    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 29 18:18:10.886842 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.886777    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 29 18:18:10.886842 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.886792    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 29 18:18:10.886842 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.886803    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-m2c66\""
May 29 18:18:10.894691 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.894665    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"]
May 29 18:18:10.995139 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.995102    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/805a7490-9c21-4229-ba97-34db7ada8a81-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:10.995139 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.995136    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/805a7490-9c21-4229-ba97-34db7ada8a81-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:10.995370 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.995243    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:10.995370 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.995294    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df5pl\" (UniqueName: \"kubernetes.io/projected/805a7490-9c21-4229-ba97-34db7ada8a81-kube-api-access-df5pl\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:10.995434 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.995386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:10.995434 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:10.995420    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.096544 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.096515    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.096731 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.096558    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.096731 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.096579    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/805a7490-9c21-4229-ba97-34db7ada8a81-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.096731 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.096600    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/805a7490-9c21-4229-ba97-34db7ada8a81-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.096731 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.096630    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.096731 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.096653    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-df5pl\" (UniqueName: \"kubernetes.io/projected/805a7490-9c21-4229-ba97-34db7ada8a81-kube-api-access-df5pl\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.097112 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.097084    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/805a7490-9c21-4229-ba97-34db7ada8a81-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.097463 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.097437    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/805a7490-9c21-4229-ba97-34db7ada8a81-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.099150 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.099124    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.099257 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.099185    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.099257 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.099208    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.105450 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.105429    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-df5pl\" (UniqueName: \"kubernetes.io/projected/805a7490-9c21-4229-ba97-34db7ada8a81-kube-api-access-df5pl\") pod \"feast-simple-feast-remote-setup-5cff94746b-f9b7n\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.200721 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.200636    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:11.327598 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.327569    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"]
May 29 18:18:11.329434 ip-10-0-133-167 kubenswrapper[2575]: W0529 18:18:11.329408    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod805a7490_9c21_4229_ba97_34db7ada8a81.slice/crio-89f1cd84b5429bf7da247337cb13a1cb752ed81f02cef97fdd8d4ad869f85c97 WatchSource:0}: Error finding container 89f1cd84b5429bf7da247337cb13a1cb752ed81f02cef97fdd8d4ad869f85c97: Status 404 returned error can't find the container with id 89f1cd84b5429bf7da247337cb13a1cb752ed81f02cef97fdd8d4ad869f85c97
May 29 18:18:11.509046 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.508965    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerStarted","Data":"89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746"}
May 29 18:18:11.509046 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:11.509000    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerStarted","Data":"89f1cd84b5429bf7da247337cb13a1cb752ed81f02cef97fdd8d4ad869f85c97"}
May 29 18:18:16.524484 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:16.524452    2575 generic.go:358] "Generic (PLEG): container finished" podID="805a7490-9c21-4229-ba97-34db7ada8a81" containerID="89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746" exitCode=0
May 29 18:18:16.524866 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:16.524496    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerDied","Data":"89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746"}
May 29 18:18:17.530065 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:17.530026    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerStarted","Data":"43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf"}
May 29 18:18:22.549569 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:22.549535    2575 generic.go:358] "Generic (PLEG): container finished" podID="805a7490-9c21-4229-ba97-34db7ada8a81" containerID="43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf" exitCode=0
May 29 18:18:22.549949 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:22.549604    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerDied","Data":"43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf"}
May 29 18:18:23.555970 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:23.555935    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerStarted","Data":"71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868"}
May 29 18:18:23.555970 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:23.555974    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerStarted","Data":"772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24"}
May 29 18:18:23.556520 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:23.555984    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerStarted","Data":"888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9"}
May 29 18:18:23.583388 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:23.583335    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podStartSLOduration=13.583319252999999 podStartE2EDuration="13.583319253s" podCreationTimestamp="2026-05-29 18:18:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:18:23.580917057 +0000 UTC m=+1278.636710420" watchObservedRunningTime="2026-05-29 18:18:23.583319253 +0000 UTC m=+1278.639112617"
May 29 18:18:26.201813 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:26.201367    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:26.203340 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:26.202880    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:26.203340 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:26.202909    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:26.203340 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:26.203277    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 29 18:18:26.203636 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:26.203611    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 29 18:18:26.203695 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:26.203673    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused"
May 29 18:18:29.202474 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:29.202443    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:29.202985 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:29.202631    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:29.203054 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:29.202983    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:29.203054 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:29.203015    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:29.203280 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:29.203260    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:29.203351 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:29.203318    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:29.206475 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:29.206454    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:29.577249 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:29.577145    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:18:29.580016 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:18:29.579993    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:19:32.082966 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.082873    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"]
May 29 18:19:32.083494 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.083335    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="online" containerID="cri-o://888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9" gracePeriod=30
May 29 18:19:32.083494 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.083400    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="offline" containerID="cri-o://772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24" gracePeriod=30
May 29 18:19:32.083494 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.083375    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="ui" containerID="cri-o://71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868" gracePeriod=30
May 29 18:19:32.225604 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.225571    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"]
May 29 18:19:32.226085 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.225906    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="registry" containerID="cri-o://84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0" gracePeriod=30
May 29 18:19:32.226085 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.225954    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="ui" containerID="cri-o://897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c" gracePeriod=30
May 29 18:19:32.226085 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.225982    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="offline" containerID="cri-o://a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4" gracePeriod=30
May 29 18:19:32.226085 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.226007    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="online" containerID="cri-o://49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce" gracePeriod=30
May 29 18:19:32.775278 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.775249    2575 generic.go:358] "Generic (PLEG): container finished" podID="805a7490-9c21-4229-ba97-34db7ada8a81" containerID="71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868" exitCode=0
May 29 18:19:32.775389 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.775312    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerDied","Data":"71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868"}
May 29 18:19:32.777754 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.777732    2575 generic.go:358] "Generic (PLEG): container finished" podID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerID="897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c" exitCode=0
May 29 18:19:32.777852 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:32.777796    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerDied","Data":"897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c"}
May 29 18:19:33.782549 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:33.782517    2575 generic.go:358] "Generic (PLEG): container finished" podID="805a7490-9c21-4229-ba97-34db7ada8a81" containerID="888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9" exitCode=0
May 29 18:19:33.782978 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:33.782589    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerDied","Data":"888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9"}
May 29 18:19:33.785026 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:33.785003    2575 generic.go:358] "Generic (PLEG): container finished" podID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerID="49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce" exitCode=0
May 29 18:19:33.785139 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:33.785034    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerDied","Data":"49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce"}
May 29 18:19:37.415870 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:37.415824    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 29 18:19:38.313878 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:38.313835    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 29 18:19:39.203835 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:39.203769    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 29 18:19:39.577809 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:39.577717    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 29 18:19:47.415988 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:47.415934    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 29 18:19:48.314557 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:48.314508    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 29 18:19:49.203262 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:49.203209    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 29 18:19:49.578101 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:49.578011    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 29 18:19:57.416455 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:57.416400    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 29 18:19:57.416890 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:57.416558    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:19:58.314504 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:58.314464    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 29 18:19:58.314683 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:58.314586    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:19:59.203230 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:59.203189    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 29 18:19:59.203680 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:59.203323    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:19:59.577906 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:59.577804    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 29 18:19:59.578077 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:19:59.577942    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:20:02.340432 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.340363    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:20:02.370860 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.370827    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-offline-tls\") pod \"805a7490-9c21-4229-ba97-34db7ada8a81\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") "
May 29 18:20:02.371026 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.370911    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/805a7490-9c21-4229-ba97-34db7ada8a81-registry-tls\") pod \"805a7490-9c21-4229-ba97-34db7ada8a81\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") "
May 29 18:20:02.371026 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.370938    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-ui-tls\") pod \"805a7490-9c21-4229-ba97-34db7ada8a81\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") "
May 29 18:20:02.371026 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.370969    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/805a7490-9c21-4229-ba97-34db7ada8a81-feast-data\") pod \"805a7490-9c21-4229-ba97-34db7ada8a81\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") "
May 29 18:20:02.371026 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.370989    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-online-tls\") pod \"805a7490-9c21-4229-ba97-34db7ada8a81\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") "
May 29 18:20:02.371026 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.371014    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-df5pl\" (UniqueName: \"kubernetes.io/projected/805a7490-9c21-4229-ba97-34db7ada8a81-kube-api-access-df5pl\") pod \"805a7490-9c21-4229-ba97-34db7ada8a81\" (UID: \"805a7490-9c21-4229-ba97-34db7ada8a81\") "
May 29 18:20:02.371360 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.371335    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/805a7490-9c21-4229-ba97-34db7ada8a81-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "805a7490-9c21-4229-ba97-34db7ada8a81" (UID: "805a7490-9c21-4229-ba97-34db7ada8a81"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:20:02.371540 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.371514    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/805a7490-9c21-4229-ba97-34db7ada8a81-feast-data" (OuterVolumeSpecName: "feast-data") pod "805a7490-9c21-4229-ba97-34db7ada8a81" (UID: "805a7490-9c21-4229-ba97-34db7ada8a81"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:20:02.373189 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.373147    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-online-tls" (OuterVolumeSpecName: "online-tls") pod "805a7490-9c21-4229-ba97-34db7ada8a81" (UID: "805a7490-9c21-4229-ba97-34db7ada8a81"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:20:02.373425 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.373405    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "805a7490-9c21-4229-ba97-34db7ada8a81" (UID: "805a7490-9c21-4229-ba97-34db7ada8a81"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:20:02.373550 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.373535    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "805a7490-9c21-4229-ba97-34db7ada8a81" (UID: "805a7490-9c21-4229-ba97-34db7ada8a81"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:20:02.373617 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.373574    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805a7490-9c21-4229-ba97-34db7ada8a81-kube-api-access-df5pl" (OuterVolumeSpecName: "kube-api-access-df5pl") pod "805a7490-9c21-4229-ba97-34db7ada8a81" (UID: "805a7490-9c21-4229-ba97-34db7ada8a81"). InnerVolumeSpecName "kube-api-access-df5pl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:20:02.471702 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.471665    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/805a7490-9c21-4229-ba97-34db7ada8a81-registry-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.471702 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.471695    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-ui-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.471702 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.471705    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/805a7490-9c21-4229-ba97-34db7ada8a81-feast-data\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.471702 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.471714    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-online-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.471971 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.471723    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-df5pl\" (UniqueName: \"kubernetes.io/projected/805a7490-9c21-4229-ba97-34db7ada8a81-kube-api-access-df5pl\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.471971 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.471732    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/805a7490-9c21-4229-ba97-34db7ada8a81-offline-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.868509 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.868478    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:20:02.874533 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.874514    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6knf\" (UniqueName: \"kubernetes.io/projected/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-kube-api-access-w6knf\") pod \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") "
May 29 18:20:02.874641 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.874558    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-feast-data\") pod \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") "
May 29 18:20:02.874641 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.874590    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-online-tls\") pod \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") "
May 29 18:20:02.874641 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.874637    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-offline-tls\") pod \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") "
May 29 18:20:02.874794 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.874676    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-ui-tls\") pod \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") "
May 29 18:20:02.874794 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.874737    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-registry-tls\") pod \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\" (UID: \"8c9747d6-ae55-48e1-9ead-c8d0c12c0883\") "
May 29 18:20:02.875218 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.875086    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-feast-data" (OuterVolumeSpecName: "feast-data") pod "8c9747d6-ae55-48e1-9ead-c8d0c12c0883" (UID: "8c9747d6-ae55-48e1-9ead-c8d0c12c0883"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:20:02.876851 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.876826    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-online-tls" (OuterVolumeSpecName: "online-tls") pod "8c9747d6-ae55-48e1-9ead-c8d0c12c0883" (UID: "8c9747d6-ae55-48e1-9ead-c8d0c12c0883"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:20:02.877022 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.876989    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8c9747d6-ae55-48e1-9ead-c8d0c12c0883" (UID: "8c9747d6-ae55-48e1-9ead-c8d0c12c0883"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:20:02.877295 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.877254    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "8c9747d6-ae55-48e1-9ead-c8d0c12c0883" (UID: "8c9747d6-ae55-48e1-9ead-c8d0c12c0883"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:20:02.877295 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.877263    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-kube-api-access-w6knf" (OuterVolumeSpecName: "kube-api-access-w6knf") pod "8c9747d6-ae55-48e1-9ead-c8d0c12c0883" (UID: "8c9747d6-ae55-48e1-9ead-c8d0c12c0883"). InnerVolumeSpecName "kube-api-access-w6knf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:20:02.877295 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.877266    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "8c9747d6-ae55-48e1-9ead-c8d0c12c0883" (UID: "8c9747d6-ae55-48e1-9ead-c8d0c12c0883"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:20:02.879591 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.879570    2575 generic.go:358] "Generic (PLEG): container finished" podID="805a7490-9c21-4229-ba97-34db7ada8a81" containerID="772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24" exitCode=137
May 29 18:20:02.879695 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.879638    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerDied","Data":"772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24"}
May 29 18:20:02.879695 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.879677    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n" event={"ID":"805a7490-9c21-4229-ba97-34db7ada8a81","Type":"ContainerDied","Data":"89f1cd84b5429bf7da247337cb13a1cb752ed81f02cef97fdd8d4ad869f85c97"}
May 29 18:20:02.879800 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.879699    2575 scope.go:117] "RemoveContainer" containerID="71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868"
May 29 18:20:02.879800 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.879649    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"
May 29 18:20:02.882438 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.882397    2575 generic.go:358] "Generic (PLEG): container finished" podID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerID="a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4" exitCode=137
May 29 18:20:02.882554 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.882440    2575 generic.go:358] "Generic (PLEG): container finished" podID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerID="84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0" exitCode=137
May 29 18:20:02.882554 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.882463    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"
May 29 18:20:02.882554 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.882470    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerDied","Data":"a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4"}
May 29 18:20:02.882554 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.882507    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerDied","Data":"84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0"}
May 29 18:20:02.882554 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.882520    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv" event={"ID":"8c9747d6-ae55-48e1-9ead-c8d0c12c0883","Type":"ContainerDied","Data":"2c73b476a0f5d362a7ccdb4575160e56a29b66111470bbe09e09b3551fda9269"}
May 29 18:20:02.888905 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.888820    2575 scope.go:117] "RemoveContainer" containerID="772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24"
May 29 18:20:02.900126 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.900104    2575 scope.go:117] "RemoveContainer" containerID="888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9"
May 29 18:20:02.909195 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.909169    2575 scope.go:117] "RemoveContainer" containerID="43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf"
May 29 18:20:02.911445 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.911396    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"]
May 29 18:20:02.914550 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.914530    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-f9b7n"]
May 29 18:20:02.917857 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.917838    2575 scope.go:117] "RemoveContainer" containerID="89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746"
May 29 18:20:02.925875 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.925851    2575 scope.go:117] "RemoveContainer" containerID="71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868"
May 29 18:20:02.926143 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.926124    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868\": container with ID starting with 71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868 not found: ID does not exist" containerID="71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868"
May 29 18:20:02.926254 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.926193    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868"} err="failed to get container status \"71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868\": rpc error: code = NotFound desc = could not find container \"71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868\": container with ID starting with 71c371163a3ecc694bb31411b9fd5ba41e63a76a8c0f159c529a0994de93d868 not found: ID does not exist"
May 29 18:20:02.926254 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.926233    2575 scope.go:117] "RemoveContainer" containerID="772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24"
May 29 18:20:02.926456 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.926439    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"]
May 29 18:20:02.926497 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.926476    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24\": container with ID starting with 772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24 not found: ID does not exist" containerID="772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24"
May 29 18:20:02.926531 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.926496    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24"} err="failed to get container status \"772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24\": rpc error: code = NotFound desc = could not find container \"772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24\": container with ID starting with 772adcc0e6d0428ca425ef04473ed978af409631b148e7e9aaedda84dd0fbd24 not found: ID does not exist"
May 29 18:20:02.926531 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.926509    2575 scope.go:117] "RemoveContainer" containerID="888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9"
May 29 18:20:02.926747 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.926731    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9\": container with ID starting with 888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9 not found: ID does not exist" containerID="888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9"
May 29 18:20:02.926798 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.926752    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9"} err="failed to get container status \"888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9\": rpc error: code = NotFound desc = could not find container \"888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9\": container with ID starting with 888f853d0503d0c2f54babc44eb687aa73244f49894a7d78106b3431dbd10dd9 not found: ID does not exist"
May 29 18:20:02.926798 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.926767    2575 scope.go:117] "RemoveContainer" containerID="43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf"
May 29 18:20:02.926977 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.926962    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf\": container with ID starting with 43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf not found: ID does not exist" containerID="43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf"
May 29 18:20:02.927021 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.926980    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf"} err="failed to get container status \"43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf\": rpc error: code = NotFound desc = could not find container \"43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf\": container with ID starting with 43f85bd76ba5ec6709e1404463ff617309c94975bdccc5e8e973a7f7be42deaf not found: ID does not exist"
May 29 18:20:02.927021 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.926992    2575 scope.go:117] "RemoveContainer" containerID="89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746"
May 29 18:20:02.927238 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.927221    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746\": container with ID starting with 89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746 not found: ID does not exist" containerID="89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746"
May 29 18:20:02.927294 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.927243    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746"} err="failed to get container status \"89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746\": rpc error: code = NotFound desc = could not find container \"89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746\": container with ID starting with 89f120969dc71c94e654bb7a9fb6359e6306cc3b278d1bb88aaaf79a4e777746 not found: ID does not exist"
May 29 18:20:02.927294 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.927256    2575 scope.go:117] "RemoveContainer" containerID="897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c"
May 29 18:20:02.931930 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.931910    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-l2hbv"]
May 29 18:20:02.935262 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.935242    2575 scope.go:117] "RemoveContainer" containerID="a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4"
May 29 18:20:02.942101 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.942084    2575 scope.go:117] "RemoveContainer" containerID="49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce"
May 29 18:20:02.948764 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.948750    2575 scope.go:117] "RemoveContainer" containerID="84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0"
May 29 18:20:02.955564 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.955546    2575 scope.go:117] "RemoveContainer" containerID="4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15"
May 29 18:20:02.962495 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.962479    2575 scope.go:117] "RemoveContainer" containerID="44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7"
May 29 18:20:02.969095 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.969073    2575 scope.go:117] "RemoveContainer" containerID="897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c"
May 29 18:20:02.969378 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.969358    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c\": container with ID starting with 897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c not found: ID does not exist" containerID="897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c"
May 29 18:20:02.969445 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.969387    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c"} err="failed to get container status \"897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c\": rpc error: code = NotFound desc = could not find container \"897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c\": container with ID starting with 897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c not found: ID does not exist"
May 29 18:20:02.969445 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.969407    2575 scope.go:117] "RemoveContainer" containerID="a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4"
May 29 18:20:02.969654 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.969639    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4\": container with ID starting with a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4 not found: ID does not exist" containerID="a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4"
May 29 18:20:02.969714 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.969655    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4"} err="failed to get container status \"a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4\": rpc error: code = NotFound desc = could not find container \"a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4\": container with ID starting with a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4 not found: ID does not exist"
May 29 18:20:02.969714 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.969669    2575 scope.go:117] "RemoveContainer" containerID="49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce"
May 29 18:20:02.969868 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.969851    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce\": container with ID starting with 49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce not found: ID does not exist" containerID="49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce"
May 29 18:20:02.969908 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.969873    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce"} err="failed to get container status \"49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce\": rpc error: code = NotFound desc = could not find container \"49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce\": container with ID starting with 49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce not found: ID does not exist"
May 29 18:20:02.969908 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.969888    2575 scope.go:117] "RemoveContainer" containerID="84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0"
May 29 18:20:02.970131 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.970118    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0\": container with ID starting with 84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0 not found: ID does not exist" containerID="84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0"
May 29 18:20:02.970190 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.970134    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0"} err="failed to get container status \"84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0\": rpc error: code = NotFound desc = could not find container \"84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0\": container with ID starting with 84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0 not found: ID does not exist"
May 29 18:20:02.970190 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.970145    2575 scope.go:117] "RemoveContainer" containerID="4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15"
May 29 18:20:02.970487 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.970470    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15\": container with ID starting with 4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15 not found: ID does not exist" containerID="4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15"
May 29 18:20:02.970530 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.970491    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15"} err="failed to get container status \"4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15\": rpc error: code = NotFound desc = could not find container \"4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15\": container with ID starting with 4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15 not found: ID does not exist"
May 29 18:20:02.970530 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.970504    2575 scope.go:117] "RemoveContainer" containerID="44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7"
May 29 18:20:02.970696 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:20:02.970678    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7\": container with ID starting with 44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7 not found: ID does not exist" containerID="44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7"
May 29 18:20:02.970736 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.970698    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7"} err="failed to get container status \"44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7\": rpc error: code = NotFound desc = could not find container \"44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7\": container with ID starting with 44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7 not found: ID does not exist"
May 29 18:20:02.970736 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.970710    2575 scope.go:117] "RemoveContainer" containerID="897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c"
May 29 18:20:02.970910 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.970893    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c"} err="failed to get container status \"897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c\": rpc error: code = NotFound desc = could not find container \"897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c\": container with ID starting with 897504ba9f68b557ccb8b124ab157642e0c3684effb808fb8e81efd78eee679c not found: ID does not exist"
May 29 18:20:02.970952 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.970910    2575 scope.go:117] "RemoveContainer" containerID="a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4"
May 29 18:20:02.971095 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.971078    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4"} err="failed to get container status \"a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4\": rpc error: code = NotFound desc = could not find container \"a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4\": container with ID starting with a2c551d11390890d1dbe26bff29e455af205c135ad9f500dcdc6a5e8873674c4 not found: ID does not exist"
May 29 18:20:02.971137 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.971097    2575 scope.go:117] "RemoveContainer" containerID="49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce"
May 29 18:20:02.971326 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.971306    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce"} err="failed to get container status \"49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce\": rpc error: code = NotFound desc = could not find container \"49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce\": container with ID starting with 49142defd53526877aeb293374f8f28d279cd0e0e2b935efd799ea4f099a91ce not found: ID does not exist"
May 29 18:20:02.971378 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.971326    2575 scope.go:117] "RemoveContainer" containerID="84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0"
May 29 18:20:02.971510 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.971495    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0"} err="failed to get container status \"84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0\": rpc error: code = NotFound desc = could not find container \"84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0\": container with ID starting with 84e4d8703583dae58b7d128955e91ba99bbac375a274d9570e1ece9476499bd0 not found: ID does not exist"
May 29 18:20:02.971510 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.971510    2575 scope.go:117] "RemoveContainer" containerID="4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15"
May 29 18:20:02.971699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.971684    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15"} err="failed to get container status \"4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15\": rpc error: code = NotFound desc = could not find container \"4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15\": container with ID starting with 4e22ff2d8e059cbfce28ed7f74a486a4459584651c74fed803e2873ad452cb15 not found: ID does not exist"
May 29 18:20:02.971699 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.971698    2575 scope.go:117] "RemoveContainer" containerID="44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7"
May 29 18:20:02.971871 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.971853    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7"} err="failed to get container status \"44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7\": rpc error: code = NotFound desc = could not find container \"44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7\": container with ID starting with 44995dc50796fed3669233e7d4ad8c32b08d2fc91e6b49deb9fc10b5381e35d7 not found: ID does not exist"
May 29 18:20:02.975319 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.975303    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6knf\" (UniqueName: \"kubernetes.io/projected/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-kube-api-access-w6knf\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.975391 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.975324    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-feast-data\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.975391 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.975335    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-online-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.975391 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.975343    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-offline-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.975391 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.975352    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-ui-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:02.975391 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:02.975360    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8c9747d6-ae55-48e1-9ead-c8d0c12c0883-registry-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:20:03.468277 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:03.468247    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" path="/var/lib/kubelet/pods/805a7490-9c21-4229-ba97-34db7ada8a81/volumes"
May 29 18:20:03.468805 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:03.468789    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" path="/var/lib/kubelet/pods/8c9747d6-ae55-48e1-9ead-c8d0c12c0883/volumes"
May 29 18:20:42.457602 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.457562    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"]
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.457913    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="online"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.457928    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="online"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.457947    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="feast-init"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.457955    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="feast-init"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.457962    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="ui"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.457969    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="ui"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.457982    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="feast-init"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.457990    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="feast-init"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.457998    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="online"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458003    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="online"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458018    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="registry"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458023    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="registry"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458033    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="offline"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458039    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="offline"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458044    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="offline"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458050    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="offline"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458062    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="feast-apply"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458070    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="feast-apply"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458079    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="ui"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458084    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="ui"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458099    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="feast-apply"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458106    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="feast-apply"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458188    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="registry"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458198    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="ui"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458207    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="ui"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458217    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="offline"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458226    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="online"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458237    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="805a7490-9c21-4229-ba97-34db7ada8a81" containerName="online"
May 29 18:20:42.458270 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.458249    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="8c9747d6-ae55-48e1-9ead-c8d0c12c0883" containerName="offline"
May 29 18:20:42.461859 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.461841    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.464864 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.464708    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 29 18:20:42.464864 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.464810    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 29 18:20:42.465042 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.464815    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 29 18:20:42.465042 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.464937    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 29 18:20:42.465042 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.464963    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-ntdzq\""
May 29 18:20:42.465042 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.464893    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 29 18:20:42.465042 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.464989    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 29 18:20:42.473227 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.473203    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"]
May 29 18:20:42.610047 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.610008    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.610047 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.610048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbp4c\" (UniqueName: \"kubernetes.io/projected/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-kube-api-access-nbp4c\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.610362 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.610101    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.610362 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.610266    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.610474 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.610400    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.610474 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.610435    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.711837 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.711757    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.711837 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.711792    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.711837 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.711825    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.712121 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.711845    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nbp4c\" (UniqueName: \"kubernetes.io/projected/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-kube-api-access-nbp4c\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.712121 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.711874    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.712121 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.711909    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.712320 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.712249    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.714666 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.714630    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.714842 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.714816    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.714962 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.714937    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.715226 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.715202    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.722843 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.722819    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbp4c\" (UniqueName: \"kubernetes.io/projected/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-kube-api-access-nbp4c\") pod \"feast-simple-feast-setup-664fcfcc58-grwdj\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.787627 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.787581    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:42.917720 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.917694    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"]
May 29 18:20:42.919645 ip-10-0-133-167 kubenswrapper[2575]: W0529 18:20:42.919611    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74ee25c2_7d5b_48f4_84c8_35ca1f4783a1.slice/crio-3270f83cae574660511e4bede0672348dce0ac4c4f8fe4d3c1a48eec005c8d58 WatchSource:0}: Error finding container 3270f83cae574660511e4bede0672348dce0ac4c4f8fe4d3c1a48eec005c8d58: Status 404 returned error can't find the container with id 3270f83cae574660511e4bede0672348dce0ac4c4f8fe4d3c1a48eec005c8d58
May 29 18:20:42.921308 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:42.921286    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 18:20:43.007851 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:43.007810    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerStarted","Data":"b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0"}
May 29 18:20:43.007851 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:43.007843    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerStarted","Data":"3270f83cae574660511e4bede0672348dce0ac4c4f8fe4d3c1a48eec005c8d58"}
May 29 18:20:48.025722 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:48.025688    2575 generic.go:358] "Generic (PLEG): container finished" podID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerID="b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0" exitCode=0
May 29 18:20:48.026229 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:48.025764    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerDied","Data":"b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0"}
May 29 18:20:49.031701 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:49.031618    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerStarted","Data":"7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578"}
May 29 18:20:54.053235 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:54.053197    2575 generic.go:358] "Generic (PLEG): container finished" podID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerID="7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578" exitCode=0
May 29 18:20:54.053622 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:54.053274    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerDied","Data":"7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578"}
May 29 18:20:55.062378 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:55.062337    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerStarted","Data":"c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0"}
May 29 18:20:55.062867 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:55.062391    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerStarted","Data":"e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525"}
May 29 18:20:55.062867 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:55.062407    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerStarted","Data":"c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651"}
May 29 18:20:55.062867 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:55.062420    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerStarted","Data":"22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590"}
May 29 18:20:55.088382 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:55.088321    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podStartSLOduration=13.088302837 podStartE2EDuration="13.088302837s" podCreationTimestamp="2026-05-29 18:20:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:20:55.085124667 +0000 UTC m=+1430.140918056" watchObservedRunningTime="2026-05-29 18:20:55.088302837 +0000 UTC m=+1430.144096202"
May 29 18:20:57.788238 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:57.788198    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:57.788726 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:57.788256    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:57.788726 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:57.788271    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:57.788726 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:57.788284    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:20:57.790288 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:57.790245    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused"
May 29 18:20:57.790424 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:57.790283    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 29 18:20:57.790424 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:57.790249    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused"
May 29 18:20:57.790591 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:20:57.790557    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 29 18:21:00.788640 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:00.788599    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:00.789086 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:00.788777    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:00.789135 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:00.789091    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:00.789544 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:00.789523    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:00.789655 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:00.789550    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:00.789655 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:00.789562    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:00.789655 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:00.789620    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:00.789773 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:00.789663    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:00.789773 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:00.789701    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:00.793342 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:00.793324    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:01.085475 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:01.085393    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:21:01.088268 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:21:01.088248    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:22:03.396651 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.396610    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"]
May 29 18:22:03.397213 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.397111    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="registry" containerID="cri-o://22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590" gracePeriod=30
May 29 18:22:03.397295 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.397186    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="ui" containerID="cri-o://c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0" gracePeriod=30
May 29 18:22:03.397295 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.397269    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="offline" containerID="cri-o://e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525" gracePeriod=30
May 29 18:22:03.397411 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.397343    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="online" containerID="cri-o://c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651" gracePeriod=30
May 29 18:22:03.697225 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.697118    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"]
May 29 18:22:03.701248 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.701227    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.704769 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.704746    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-9nfm6\""
May 29 18:22:03.715765 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.715739    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"]
May 29 18:22:03.721146 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.721118    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.721300 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.721222    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.721300 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.721264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.721430 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.721311    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/174801ae-50d3-48a8-9fd7-1de9d580e17b-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.721430 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.721342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99gkn\" (UniqueName: \"kubernetes.io/projected/174801ae-50d3-48a8-9fd7-1de9d580e17b-kube-api-access-99gkn\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.721430 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.721418    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.822602 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.822570    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.822734 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.822612    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.822734 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.822644    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/174801ae-50d3-48a8-9fd7-1de9d580e17b-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.822734 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.822665    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-99gkn\" (UniqueName: \"kubernetes.io/projected/174801ae-50d3-48a8-9fd7-1de9d580e17b-kube-api-access-99gkn\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.822734 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.822695    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.822734 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:22:03.822704    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 29 18:22:03.822734 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.822734    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.823036 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:22:03.822778    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-ui-tls podName:174801ae-50d3-48a8-9fd7-1de9d580e17b nodeName:}" failed. No retries permitted until 2026-05-29 18:22:04.322756879 +0000 UTC m=+1499.378550236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-2x4wb" (UID: "174801ae-50d3-48a8-9fd7-1de9d580e17b") : secret "feast-simple-feast-setup-ui-tls" not found
May 29 18:22:03.823036 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:22:03.822815    2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 29 18:22:03.823036 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:22:03.822868    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-offline-tls podName:174801ae-50d3-48a8-9fd7-1de9d580e17b nodeName:}" failed. No retries permitted until 2026-05-29 18:22:04.322849756 +0000 UTC m=+1499.378643096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-2x4wb" (UID: "174801ae-50d3-48a8-9fd7-1de9d580e17b") : secret "feast-simple-feast-setup-offline-tls" not found
May 29 18:22:03.823036 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.823010    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/174801ae-50d3-48a8-9fd7-1de9d580e17b-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.825168 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.825129    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.825288 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.825180    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:03.831685 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:03.831663    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99gkn\" (UniqueName: \"kubernetes.io/projected/174801ae-50d3-48a8-9fd7-1de9d580e17b-kube-api-access-99gkn\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:04.279788 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:04.279703    2575 generic.go:358] "Generic (PLEG): container finished" podID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerID="c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0" exitCode=0
May 29 18:22:04.279788 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:04.279728    2575 generic.go:358] "Generic (PLEG): container finished" podID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerID="c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651" exitCode=0
May 29 18:22:04.279788 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:04.279771    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerDied","Data":"c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0"}
May 29 18:22:04.279995 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:04.279801    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerDied","Data":"c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651"}
May 29 18:22:04.327841 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:04.327812    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:04.327974 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:04.327875    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:04.330179 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:04.330124    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:04.330179 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:04.330144    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-2x4wb\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:04.611220 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:04.611111    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:04.747652 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:04.747622    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"]
May 29 18:22:04.748654 ip-10-0-133-167 kubenswrapper[2575]: W0529 18:22:04.748633    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod174801ae_50d3_48a8_9fd7_1de9d580e17b.slice/crio-88d2677bfd5a23413352a9883106c43aeaf1e513bdd5a5ca3d0952ee08f5371f WatchSource:0}: Error finding container 88d2677bfd5a23413352a9883106c43aeaf1e513bdd5a5ca3d0952ee08f5371f: Status 404 returned error can't find the container with id 88d2677bfd5a23413352a9883106c43aeaf1e513bdd5a5ca3d0952ee08f5371f
May 29 18:22:05.284141 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:05.284107    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerStarted","Data":"ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41"}
May 29 18:22:05.284141 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:05.284143    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerStarted","Data":"88d2677bfd5a23413352a9883106c43aeaf1e513bdd5a5ca3d0952ee08f5371f"}
May 29 18:22:05.514360 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:05.514330    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:22:05.517628 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:05.517609    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:22:10.301716 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:10.301679    2575 generic.go:358] "Generic (PLEG): container finished" podID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerID="ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41" exitCode=0
May 29 18:22:10.302133 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:10.301720    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerDied","Data":"ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41"}
May 29 18:22:10.790070 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:10.790022    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 29 18:22:11.085908 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:11.085819    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 29 18:22:11.306948 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:11.306915    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerStarted","Data":"707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62"}
May 29 18:22:16.324928 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:16.324891    2575 generic.go:358] "Generic (PLEG): container finished" podID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerID="707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62" exitCode=0
May 29 18:22:16.324928 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:16.324930    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerDied","Data":"707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62"}
May 29 18:22:17.332572 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:17.332532    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerStarted","Data":"53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c"}
May 29 18:22:17.333055 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:17.332583    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerStarted","Data":"080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a"}
May 29 18:22:17.333055 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:17.332598    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerStarted","Data":"b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655"}
May 29 18:22:17.333055 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:17.332612    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerStarted","Data":"abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623"}
May 29 18:22:17.359050 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:17.358977    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podStartSLOduration=14.358956425 podStartE2EDuration="14.358956425s" podCreationTimestamp="2026-05-29 18:22:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:22:17.356527038 +0000 UTC m=+1512.412320395" watchObservedRunningTime="2026-05-29 18:22:17.358956425 +0000 UTC m=+1512.414749841"
May 29 18:22:19.611989 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:19.611893    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:19.611989 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:19.611949    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:19.611989 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:19.611964    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:19.611989 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:19.611975    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:19.614185 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:19.614140    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 29 18:22:19.614434 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:19.614183    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused"
May 29 18:22:19.614563 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:19.614199    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused"
May 29 18:22:19.614563 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:19.614249    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 29 18:22:20.790393 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:20.790342    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 29 18:22:21.085862 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:21.085741    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 29 18:22:22.612306 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:22.612274    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:22.612669 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:22.612423    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:22.612669 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:22.612578    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:22.613120 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:22.613091    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:22.613286 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:22.613131    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:22.613286 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:22.613145    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:22.613439 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:22.613368    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:22.613439 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:22.613409    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:22.613566 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:22.613548    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:22.616986 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:22.616971    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:23.355710 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:23.355677    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:23.358588 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:23.358569    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:22:30.789940 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:30.789888    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 29 18:22:30.790500 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:30.790018    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:22:31.085949 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:31.085861    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 29 18:22:31.086097 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:31.086010    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:22:34.048325 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.048303    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:22:34.207040 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.206932    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-registry-tls\") pod \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") "
May 29 18:22:34.207040 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.207009    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-offline-tls\") pod \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") "
May 29 18:22:34.207314 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.207084    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-online-tls\") pod \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") "
May 29 18:22:34.207314 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.207109    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-ui-tls\") pod \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") "
May 29 18:22:34.207314 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.207200    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-feast-data\") pod \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") "
May 29 18:22:34.207314 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.207233    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbp4c\" (UniqueName: \"kubernetes.io/projected/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-kube-api-access-nbp4c\") pod \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\" (UID: \"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1\") "
May 29 18:22:34.207834 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.207763    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-feast-data" (OuterVolumeSpecName: "feast-data") pod "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" (UID: "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:22:34.209416 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.209387    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-online-tls" (OuterVolumeSpecName: "online-tls") pod "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" (UID: "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:22:34.209539 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.209513    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" (UID: "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:22:34.209587 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.209535    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" (UID: "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:22:34.209587 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.209542    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-kube-api-access-nbp4c" (OuterVolumeSpecName: "kube-api-access-nbp4c") pod "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" (UID: "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1"). InnerVolumeSpecName "kube-api-access-nbp4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:22:34.209655 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.209630    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" (UID: "74ee25c2-7d5b-48f4-84c8-35ca1f4783a1"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:22:34.308909 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.308878    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-online-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:22:34.308909 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.308903    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-ui-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:22:34.308909 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.308912    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-feast-data\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:22:34.309127 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.308921    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbp4c\" (UniqueName: \"kubernetes.io/projected/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-kube-api-access-nbp4c\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:22:34.309127 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.308930    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-registry-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:22:34.309127 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.308938    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1-offline-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:22:34.392931 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.392896    2575 generic.go:358] "Generic (PLEG): container finished" podID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerID="e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525" exitCode=137
May 29 18:22:34.392931 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.392921    2575 generic.go:358] "Generic (PLEG): container finished" podID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerID="22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590" exitCode=137
May 29 18:22:34.393127 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.392958    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"
May 29 18:22:34.393127 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.392976    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerDied","Data":"e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525"}
May 29 18:22:34.393127 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.393009    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerDied","Data":"22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590"}
May 29 18:22:34.393127 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.393020    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj" event={"ID":"74ee25c2-7d5b-48f4-84c8-35ca1f4783a1","Type":"ContainerDied","Data":"3270f83cae574660511e4bede0672348dce0ac4c4f8fe4d3c1a48eec005c8d58"}
May 29 18:22:34.393127 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.393035    2575 scope.go:117] "RemoveContainer" containerID="c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0"
May 29 18:22:34.402163 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.402133    2575 scope.go:117] "RemoveContainer" containerID="e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525"
May 29 18:22:34.413960 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.413942    2575 scope.go:117] "RemoveContainer" containerID="c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651"
May 29 18:22:34.416833 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.416779    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"]
May 29 18:22:34.421268 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.421244    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-grwdj"]
May 29 18:22:34.422998 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.422986    2575 scope.go:117] "RemoveContainer" containerID="22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590"
May 29 18:22:34.429947 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.429929    2575 scope.go:117] "RemoveContainer" containerID="7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578"
May 29 18:22:34.436574 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.436556    2575 scope.go:117] "RemoveContainer" containerID="b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0"
May 29 18:22:34.443251 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.443210    2575 scope.go:117] "RemoveContainer" containerID="c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0"
May 29 18:22:34.443455 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:22:34.443437    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0\": container with ID starting with c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0 not found: ID does not exist" containerID="c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0"
May 29 18:22:34.443504 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.443463    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0"} err="failed to get container status \"c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0\": rpc error: code = NotFound desc = could not find container \"c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0\": container with ID starting with c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0 not found: ID does not exist"
May 29 18:22:34.443504 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.443485    2575 scope.go:117] "RemoveContainer" containerID="e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525"
May 29 18:22:34.443709 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:22:34.443691    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525\": container with ID starting with e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525 not found: ID does not exist" containerID="e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525"
May 29 18:22:34.443756 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.443716    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525"} err="failed to get container status \"e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525\": rpc error: code = NotFound desc = could not find container \"e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525\": container with ID starting with e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525 not found: ID does not exist"
May 29 18:22:34.443756 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.443734    2575 scope.go:117] "RemoveContainer" containerID="c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651"
May 29 18:22:34.443980 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:22:34.443962    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651\": container with ID starting with c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651 not found: ID does not exist" containerID="c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651"
May 29 18:22:34.444044 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.443991    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651"} err="failed to get container status \"c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651\": rpc error: code = NotFound desc = could not find container \"c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651\": container with ID starting with c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651 not found: ID does not exist"
May 29 18:22:34.444044 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.444013    2575 scope.go:117] "RemoveContainer" containerID="22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590"
May 29 18:22:34.444259 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:22:34.444241    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590\": container with ID starting with 22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590 not found: ID does not exist" containerID="22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590"
May 29 18:22:34.444318 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.444263    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590"} err="failed to get container status \"22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590\": rpc error: code = NotFound desc = could not find container \"22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590\": container with ID starting with 22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590 not found: ID does not exist"
May 29 18:22:34.444318 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.444278    2575 scope.go:117] "RemoveContainer" containerID="7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578"
May 29 18:22:34.444503 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:22:34.444486    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578\": container with ID starting with 7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578 not found: ID does not exist" containerID="7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578"
May 29 18:22:34.444611 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.444509    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578"} err="failed to get container status \"7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578\": rpc error: code = NotFound desc = could not find container \"7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578\": container with ID starting with 7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578 not found: ID does not exist"
May 29 18:22:34.444611 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.444529    2575 scope.go:117] "RemoveContainer" containerID="b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0"
May 29 18:22:34.444749 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:22:34.444734    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0\": container with ID starting with b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0 not found: ID does not exist" containerID="b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0"
May 29 18:22:34.444790 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.444753    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0"} err="failed to get container status \"b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0\": rpc error: code = NotFound desc = could not find container \"b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0\": container with ID starting with b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0 not found: ID does not exist"
May 29 18:22:34.444790 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.444768    2575 scope.go:117] "RemoveContainer" containerID="c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0"
May 29 18:22:34.444953 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.444936    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0"} err="failed to get container status \"c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0\": rpc error: code = NotFound desc = could not find container \"c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0\": container with ID starting with c4ed1a193b3801c5b651ecc3512e1ea79e7e9b05d3622c4c04bd7e4410d3b5e0 not found: ID does not exist"
May 29 18:22:34.445015 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.444955    2575 scope.go:117] "RemoveContainer" containerID="e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525"
May 29 18:22:34.445139 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.445123    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525"} err="failed to get container status \"e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525\": rpc error: code = NotFound desc = could not find container \"e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525\": container with ID starting with e29041a2c054c04d822bf6e4744f3b2134229d7349cf0fed5c0581b309d19525 not found: ID does not exist"
May 29 18:22:34.445226 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.445141    2575 scope.go:117] "RemoveContainer" containerID="c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651"
May 29 18:22:34.445381 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.445365    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651"} err="failed to get container status \"c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651\": rpc error: code = NotFound desc = could not find container \"c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651\": container with ID starting with c8463619884e3cfade79b6fb1d9c9b5116a2ef50e344eeeb20e2f19a4e594651 not found: ID does not exist"
May 29 18:22:34.445445 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.445382    2575 scope.go:117] "RemoveContainer" containerID="22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590"
May 29 18:22:34.445601 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.445583    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590"} err="failed to get container status \"22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590\": rpc error: code = NotFound desc = could not find container \"22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590\": container with ID starting with 22e4be4468ed93f4518edaee282cecbfed6519f37010a5cf046cced1bcb8a590 not found: ID does not exist"
May 29 18:22:34.445656 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.445602    2575 scope.go:117] "RemoveContainer" containerID="7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578"
May 29 18:22:34.445790 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.445772    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578"} err="failed to get container status \"7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578\": rpc error: code = NotFound desc = could not find container \"7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578\": container with ID starting with 7e6e8d6bf8dc0aa747080fe156e5c94db872126f757127af8e31d1e1dc053578 not found: ID does not exist"
May 29 18:22:34.445857 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.445792    2575 scope.go:117] "RemoveContainer" containerID="b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0"
May 29 18:22:34.445972 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:34.445956    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0"} err="failed to get container status \"b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0\": rpc error: code = NotFound desc = could not find container \"b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0\": container with ID starting with b321bef2339ec5f945f43646d44fa6812c3dbaa627404f8103df730b35a753a0 not found: ID does not exist"
May 29 18:22:35.468413 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:22:35.468380    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" path="/var/lib/kubelet/pods/74ee25c2-7d5b-48f4-84c8-35ca1f4783a1/volumes"
May 29 18:23:25.208201 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208165    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"]
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208499    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="offline"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208509    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="offline"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208520    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="ui"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208525    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="ui"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208539    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="registry"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208544    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="registry"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208551    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="feast-init"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208556    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="feast-init"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208565    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="online"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208570    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="online"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208582    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="feast-apply"
May 29 18:23:25.208596 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208588    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="feast-apply"
May 29 18:23:25.208978 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208638    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="registry"
May 29 18:23:25.208978 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208647    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="ui"
May 29 18:23:25.208978 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208656    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="online"
May 29 18:23:25.208978 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.208662    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="74ee25c2-7d5b-48f4-84c8-35ca1f4783a1" containerName="offline"
May 29 18:23:25.213435 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.213417    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.216374 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.216348    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 29 18:23:25.216512 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.216388    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 29 18:23:25.216512 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.216453    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-vn6gc\""
May 29 18:23:25.216636 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.216618    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 29 18:23:25.216794 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.216779    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 29 18:23:25.216876 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.216838    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 29 18:23:25.217412 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.217390    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 29 18:23:25.224088 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.224067    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"]
May 29 18:23:25.352400 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.352362    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.352400 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.352402    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.352623 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.352465    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.352623 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.352505    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfcgz\" (UniqueName: \"kubernetes.io/projected/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-kube-api-access-mfcgz\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.352623 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.352551    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.352623 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.352569    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.453957 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.453920    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.453957 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.453959    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.454150 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:23:25.454065    2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found
May 29 18:23:25.454150 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.454074    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.454150 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:23:25.454142    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls podName:b0a1617f-fdae-4f8e-a72c-fbb5f118a54b nodeName:}" failed. No retries permitted until 2026-05-29 18:23:25.954118732 +0000 UTC m=+1581.009912090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls") pod "feast-simple-feast-remote-setup-5cff94746b-s5jkn" (UID: "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b") : secret "feast-simple-feast-remote-setup-ui-tls" not found
May 29 18:23:25.454306 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.454214    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mfcgz\" (UniqueName: \"kubernetes.io/projected/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-kube-api-access-mfcgz\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.454306 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.454260    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.454306 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.454289    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.454683 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.454638    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.454885 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.454867    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.456391 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.456372    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.456487 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.456435    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.466006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.465933    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfcgz\" (UniqueName: \"kubernetes.io/projected/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-kube-api-access-mfcgz\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.958169 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:25.958121    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:25.958358 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:23:25.958260    2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found
May 29 18:23:25.958358 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:23:25.958332    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls podName:b0a1617f-fdae-4f8e-a72c-fbb5f118a54b nodeName:}" failed. No retries permitted until 2026-05-29 18:23:26.958316294 +0000 UTC m=+1582.014109634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls") pod "feast-simple-feast-remote-setup-5cff94746b-s5jkn" (UID: "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b") : secret "feast-simple-feast-remote-setup-ui-tls" not found
May 29 18:23:26.966504 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:26.966470    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:26.968884 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:26.968865    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-s5jkn\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:27.022831 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:27.022786    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:27.149072 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:27.149044    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"]
May 29 18:23:27.150480 ip-10-0-133-167 kubenswrapper[2575]: W0529 18:23:27.150450    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0a1617f_fdae_4f8e_a72c_fbb5f118a54b.slice/crio-b0095b3660ada5bc460b3e8763efc1c5de88b22f92e4534f24ceb1f876d1e306 WatchSource:0}: Error finding container b0095b3660ada5bc460b3e8763efc1c5de88b22f92e4534f24ceb1f876d1e306: Status 404 returned error can't find the container with id b0095b3660ada5bc460b3e8763efc1c5de88b22f92e4534f24ceb1f876d1e306
May 29 18:23:27.547595 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:27.547556    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerStarted","Data":"d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630"}
May 29 18:23:27.547595 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:27.547597    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerStarted","Data":"b0095b3660ada5bc460b3e8763efc1c5de88b22f92e4534f24ceb1f876d1e306"}
May 29 18:23:32.565318 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:32.565275    2575 generic.go:358] "Generic (PLEG): container finished" podID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerID="d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630" exitCode=0
May 29 18:23:32.565709 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:32.565343    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerDied","Data":"d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630"}
May 29 18:23:33.570764 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:33.570729    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerStarted","Data":"f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400"}
May 29 18:23:38.588817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:38.588779    2575 generic.go:358] "Generic (PLEG): container finished" podID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerID="f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400" exitCode=0
May 29 18:23:38.589271 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:38.588839    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerDied","Data":"f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400"}
May 29 18:23:39.595599 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:39.595559    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerStarted","Data":"06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1"}
May 29 18:23:39.595965 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:39.595613    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerStarted","Data":"b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05"}
May 29 18:23:39.595965 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:39.595627    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerStarted","Data":"773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71"}
May 29 18:23:39.659913 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:39.659845    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podStartSLOduration=14.659823518 podStartE2EDuration="14.659823518s" podCreationTimestamp="2026-05-29 18:23:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:23:39.657906793 +0000 UTC m=+1594.713700154" watchObservedRunningTime="2026-05-29 18:23:39.659823518 +0000 UTC m=+1594.715616881"
May 29 18:23:42.023646 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:42.023608    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:42.024233 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:42.024208    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:42.024233 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:42.024236    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:42.025192 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:42.025138    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused"
May 29 18:23:42.025321 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:42.025208    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.31:8816: connect: connection refused"
May 29 18:23:42.025321 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:42.025208    2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
May 29 18:23:45.024817 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:45.024784    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:45.025311 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:45.024901    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:45.025393 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:45.025314    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:45.025393 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:45.025341    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:45.025560 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:45.025540    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:45.025781 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:45.025764    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:45.028764 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:45.028746    2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:45.617715 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:45.617684    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:23:45.620592 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:23:45.620572    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:24:48.094982 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.094944    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"]
May 29 18:24:48.095621 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.095427    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="online" containerID="cri-o://773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71" gracePeriod=30
May 29 18:24:48.095621 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.095494    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="ui" containerID="cri-o://06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1" gracePeriod=30
May 29 18:24:48.095621 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.095576    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="offline" containerID="cri-o://b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05" gracePeriod=30
May 29 18:24:48.234039 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.234002    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"]
May 29 18:24:48.234466 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.234433    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="registry" containerID="cri-o://abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623" gracePeriod=30
May 29 18:24:48.234704 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.234486    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="ui" containerID="cri-o://53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c" gracePeriod=30
May 29 18:24:48.234810 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.234486    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="offline" containerID="cri-o://080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a" gracePeriod=30
May 29 18:24:48.234810 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.234510    2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="online" containerID="cri-o://b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655" gracePeriod=30
May 29 18:24:48.827244 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.827214    2575 generic.go:358] "Generic (PLEG): container finished" podID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerID="53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c" exitCode=0
May 29 18:24:48.827388 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.827285    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerDied","Data":"53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c"}
May 29 18:24:48.829612 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.829593    2575 generic.go:358] "Generic (PLEG): container finished" podID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerID="06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1" exitCode=0
May 29 18:24:48.829612 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.829609    2575 generic.go:358] "Generic (PLEG): container finished" podID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerID="773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71" exitCode=0
May 29 18:24:48.829762 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.829632    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerDied","Data":"06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1"}
May 29 18:24:48.829762 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:48.829654    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerDied","Data":"773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71"}
May 29 18:24:49.834653 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:49.834621    2575 generic.go:358] "Generic (PLEG): container finished" podID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerID="b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655" exitCode=0
May 29 18:24:49.835009 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:49.834691    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerDied","Data":"b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655"}
May 29 18:24:52.614101 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:52.614052    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 29 18:24:53.356537 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:53.356488    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 29 18:24:55.026021 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:55.025969    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused"
May 29 18:24:55.618457 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:24:55.618412    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
May 29 18:25:02.613549 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:02.613505    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 29 18:25:03.356279 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:03.356237    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 29 18:25:05.025932 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:05.025884    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused"
May 29 18:25:05.618604 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:05.618559    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
May 29 18:25:12.613206 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:12.613128    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 29 18:25:12.613667 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:12.613308    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:25:13.356237 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:13.356188    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 29 18:25:13.356438 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:13.356308    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:25:15.026006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:15.025963    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused"
May 29 18:25:15.026460 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:15.026083    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:25:15.618708 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:15.618664    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
May 29 18:25:15.618898 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:15.618786    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:25:18.744250 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.744221    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:25:18.864766 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.864742    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-online-tls\") pod \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") "
May 29 18:25:18.865005 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.864779    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-registry-tls\") pod \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") "
May 29 18:25:18.865005 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.864810    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls\") pod \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") "
May 29 18:25:18.865005 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.864843    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-feast-data\") pod \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") "
May 29 18:25:18.865005 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.864888    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-offline-tls\") pod \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") "
May 29 18:25:18.865005 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.864954    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfcgz\" (UniqueName: \"kubernetes.io/projected/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-kube-api-access-mfcgz\") pod \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\" (UID: \"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b\") "
May 29 18:25:18.865313 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.865221    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" (UID: "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 18:25:18.865794 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.865741    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-feast-data" (OuterVolumeSpecName: "feast-data") pod "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" (UID: "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:25:18.867504 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.867468    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" (UID: "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:25:18.867504 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.867492    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-kube-api-access-mfcgz" (OuterVolumeSpecName: "kube-api-access-mfcgz") pod "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" (UID: "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b"). InnerVolumeSpecName "kube-api-access-mfcgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:25:18.867637 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.867506    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-online-tls" (OuterVolumeSpecName: "online-tls") pod "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" (UID: "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:25:18.867637 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.867493    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" (UID: "b0a1617f-fdae-4f8e-a72c-fbb5f118a54b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:25:18.883502 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.883484    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:25:18.926352 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.926320    2575 generic.go:358] "Generic (PLEG): container finished" podID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerID="b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05" exitCode=137
May 29 18:25:18.926524 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.926393    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"
May 29 18:25:18.926524 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.926404    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerDied","Data":"b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05"}
May 29 18:25:18.926524 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.926443    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn" event={"ID":"b0a1617f-fdae-4f8e-a72c-fbb5f118a54b","Type":"ContainerDied","Data":"b0095b3660ada5bc460b3e8763efc1c5de88b22f92e4534f24ceb1f876d1e306"}
May 29 18:25:18.926524 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.926466    2575 scope.go:117] "RemoveContainer" containerID="06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1"
May 29 18:25:18.936117 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.936087    2575 generic.go:358] "Generic (PLEG): container finished" podID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerID="080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a" exitCode=137
May 29 18:25:18.936117 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.936116    2575 generic.go:358] "Generic (PLEG): container finished" podID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerID="abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623" exitCode=137
May 29 18:25:18.936338 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.936189    2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"
May 29 18:25:18.936398 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.936190    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerDied","Data":"080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a"}
May 29 18:25:18.936449 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.936395    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerDied","Data":"abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623"}
May 29 18:25:18.936449 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.936413    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb" event={"ID":"174801ae-50d3-48a8-9fd7-1de9d580e17b","Type":"ContainerDied","Data":"88d2677bfd5a23413352a9883106c43aeaf1e513bdd5a5ca3d0952ee08f5371f"}
May 29 18:25:18.943005 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.942984    2575 scope.go:117] "RemoveContainer" containerID="b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05"
May 29 18:25:18.950802 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.950784    2575 scope.go:117] "RemoveContainer" containerID="773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71"
May 29 18:25:18.957948 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.957924    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"]
May 29 18:25:18.960169 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.960135    2575 scope.go:117] "RemoveContainer" containerID="f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400"
May 29 18:25:18.960478 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.960454    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-s5jkn"]
May 29 18:25:18.966273 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966252    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/174801ae-50d3-48a8-9fd7-1de9d580e17b-feast-data\") pod \"174801ae-50d3-48a8-9fd7-1de9d580e17b\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") "
May 29 18:25:18.966369 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966284    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-registry-tls\") pod \"174801ae-50d3-48a8-9fd7-1de9d580e17b\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") "
May 29 18:25:18.966369 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966309    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99gkn\" (UniqueName: \"kubernetes.io/projected/174801ae-50d3-48a8-9fd7-1de9d580e17b-kube-api-access-99gkn\") pod \"174801ae-50d3-48a8-9fd7-1de9d580e17b\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") "
May 29 18:25:18.966369 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966358    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-offline-tls\") pod \"174801ae-50d3-48a8-9fd7-1de9d580e17b\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") "
May 29 18:25:18.966527 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966378    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-ui-tls\") pod \"174801ae-50d3-48a8-9fd7-1de9d580e17b\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") "
May 29 18:25:18.966527 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966433    2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-online-tls\") pod \"174801ae-50d3-48a8-9fd7-1de9d580e17b\" (UID: \"174801ae-50d3-48a8-9fd7-1de9d580e17b\") "
May 29 18:25:18.966702 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966682    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-offline-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:18.966811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966717    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfcgz\" (UniqueName: \"kubernetes.io/projected/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-kube-api-access-mfcgz\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:18.966811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966732    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-online-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:18.966811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966746    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-registry-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:18.966811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966760    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-ui-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:18.966811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966772    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b-feast-data\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:18.966811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.966801    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/174801ae-50d3-48a8-9fd7-1de9d580e17b-feast-data" (OuterVolumeSpecName: "feast-data") pod "174801ae-50d3-48a8-9fd7-1de9d580e17b" (UID: "174801ae-50d3-48a8-9fd7-1de9d580e17b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 18:25:18.968873 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.968843    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "174801ae-50d3-48a8-9fd7-1de9d580e17b" (UID: "174801ae-50d3-48a8-9fd7-1de9d580e17b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:25:18.969021 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.968999    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "174801ae-50d3-48a8-9fd7-1de9d580e17b" (UID: "174801ae-50d3-48a8-9fd7-1de9d580e17b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:25:18.969107 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.969056    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/174801ae-50d3-48a8-9fd7-1de9d580e17b-kube-api-access-99gkn" (OuterVolumeSpecName: "kube-api-access-99gkn") pod "174801ae-50d3-48a8-9fd7-1de9d580e17b" (UID: "174801ae-50d3-48a8-9fd7-1de9d580e17b"). InnerVolumeSpecName "kube-api-access-99gkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 18:25:18.969495 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.969471    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-online-tls" (OuterVolumeSpecName: "online-tls") pod "174801ae-50d3-48a8-9fd7-1de9d580e17b" (UID: "174801ae-50d3-48a8-9fd7-1de9d580e17b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:25:18.970024 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.969731    2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "174801ae-50d3-48a8-9fd7-1de9d580e17b" (UID: "174801ae-50d3-48a8-9fd7-1de9d580e17b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 18:25:18.970127 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.970106    2575 scope.go:117] "RemoveContainer" containerID="d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630"
May 29 18:25:18.977804 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.977787    2575 scope.go:117] "RemoveContainer" containerID="06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1"
May 29 18:25:18.978069 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:18.978050    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1\": container with ID starting with 06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1 not found: ID does not exist" containerID="06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1"
May 29 18:25:18.978126 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.978078    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1"} err="failed to get container status \"06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1\": rpc error: code = NotFound desc = could not find container \"06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1\": container with ID starting with 06effb80768deb89b9b59d6a437f51f1cf10d9713e3bbfb72c34c57ea7e4eaf1 not found: ID does not exist"
May 29 18:25:18.978126 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.978098    2575 scope.go:117] "RemoveContainer" containerID="b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05"
May 29 18:25:18.978349 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:18.978331    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05\": container with ID starting with b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05 not found: ID does not exist" containerID="b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05"
May 29 18:25:18.978391 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.978354    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05"} err="failed to get container status \"b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05\": rpc error: code = NotFound desc = could not find container \"b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05\": container with ID starting with b0e5a9050e533dde9897cf4cec5903a7fe1316441546f29e93ea6e6b042faa05 not found: ID does not exist"
May 29 18:25:18.978391 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.978370    2575 scope.go:117] "RemoveContainer" containerID="773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71"
May 29 18:25:18.978574 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:18.978559    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71\": container with ID starting with 773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71 not found: ID does not exist" containerID="773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71"
May 29 18:25:18.978630 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.978576    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71"} err="failed to get container status \"773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71\": rpc error: code = NotFound desc = could not find container \"773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71\": container with ID starting with 773a2c8c512306b9b390a424b1670097f38dda7779a050332be572f4983dea71 not found: ID does not exist"
May 29 18:25:18.978630 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.978589    2575 scope.go:117] "RemoveContainer" containerID="f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400"
May 29 18:25:18.978775 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:18.978762    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400\": container with ID starting with f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400 not found: ID does not exist" containerID="f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400"
May 29 18:25:18.978820 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.978777    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400"} err="failed to get container status \"f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400\": rpc error: code = NotFound desc = could not find container \"f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400\": container with ID starting with f62e9a59028104a4cb1c679323419c32cd02d299903acc62a8ac1ec86b970400 not found: ID does not exist"
May 29 18:25:18.978820 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.978788    2575 scope.go:117] "RemoveContainer" containerID="d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630"
May 29 18:25:18.978958 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:18.978945    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630\": container with ID starting with d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630 not found: ID does not exist" containerID="d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630"
May 29 18:25:18.978997 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.978961    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630"} err="failed to get container status \"d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630\": rpc error: code = NotFound desc = could not find container \"d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630\": container with ID starting with d5f8b559e275225e8b0bc95f1a92d58cd4668b670f26a7e9342a0d5513f2c630 not found: ID does not exist"
May 29 18:25:18.978997 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.978972    2575 scope.go:117] "RemoveContainer" containerID="53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c"
May 29 18:25:18.986104 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.986087    2575 scope.go:117] "RemoveContainer" containerID="080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a"
May 29 18:25:18.992709 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.992693    2575 scope.go:117] "RemoveContainer" containerID="b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655"
May 29 18:25:18.999693 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:18.999676    2575 scope.go:117] "RemoveContainer" containerID="abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623"
May 29 18:25:19.006334 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.006315    2575 scope.go:117] "RemoveContainer" containerID="707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62"
May 29 18:25:19.013104 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.013076    2575 scope.go:117] "RemoveContainer" containerID="ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41"
May 29 18:25:19.019640 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.019623    2575 scope.go:117] "RemoveContainer" containerID="53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c"
May 29 18:25:19.019882 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:19.019862    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c\": container with ID starting with 53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c not found: ID does not exist" containerID="53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c"
May 29 18:25:19.019947 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.019890    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c"} err="failed to get container status \"53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c\": rpc error: code = NotFound desc = could not find container \"53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c\": container with ID starting with 53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c not found: ID does not exist"
May 29 18:25:19.019947 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.019909    2575 scope.go:117] "RemoveContainer" containerID="080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a"
May 29 18:25:19.020118 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:19.020101    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a\": container with ID starting with 080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a not found: ID does not exist" containerID="080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a"
May 29 18:25:19.020178 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.020125    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a"} err="failed to get container status \"080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a\": rpc error: code = NotFound desc = could not find container \"080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a\": container with ID starting with 080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a not found: ID does not exist"
May 29 18:25:19.020178 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.020141    2575 scope.go:117] "RemoveContainer" containerID="b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655"
May 29 18:25:19.020357 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:19.020338    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655\": container with ID starting with b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655 not found: ID does not exist" containerID="b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655"
May 29 18:25:19.020414 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.020365    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655"} err="failed to get container status \"b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655\": rpc error: code = NotFound desc = could not find container \"b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655\": container with ID starting with b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655 not found: ID does not exist"
May 29 18:25:19.020414 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.020389    2575 scope.go:117] "RemoveContainer" containerID="abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623"
May 29 18:25:19.020587 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:19.020572    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623\": container with ID starting with abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623 not found: ID does not exist" containerID="abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623"
May 29 18:25:19.020626 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.020590    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623"} err="failed to get container status \"abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623\": rpc error: code = NotFound desc = could not find container \"abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623\": container with ID starting with abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623 not found: ID does not exist"
May 29 18:25:19.020626 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.020603    2575 scope.go:117] "RemoveContainer" containerID="707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62"
May 29 18:25:19.020820 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:19.020800    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62\": container with ID starting with 707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62 not found: ID does not exist" containerID="707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62"
May 29 18:25:19.020887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.020828    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62"} err="failed to get container status \"707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62\": rpc error: code = NotFound desc = could not find container \"707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62\": container with ID starting with 707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62 not found: ID does not exist"
May 29 18:25:19.020887 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.020848    2575 scope.go:117] "RemoveContainer" containerID="ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41"
May 29 18:25:19.021068 ip-10-0-133-167 kubenswrapper[2575]: E0529 18:25:19.021051    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41\": container with ID starting with ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41 not found: ID does not exist" containerID="ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41"
May 29 18:25:19.021108 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.021073    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41"} err="failed to get container status \"ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41\": rpc error: code = NotFound desc = could not find container \"ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41\": container with ID starting with ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41 not found: ID does not exist"
May 29 18:25:19.021108 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.021086    2575 scope.go:117] "RemoveContainer" containerID="53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c"
May 29 18:25:19.021422 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.021401    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c"} err="failed to get container status \"53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c\": rpc error: code = NotFound desc = could not find container \"53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c\": container with ID starting with 53ec8ea5eab99d7914f7dd782f024d52f5c940f63557c504a9a49442492de25c not found: ID does not exist"
May 29 18:25:19.021422 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.021422    2575 scope.go:117] "RemoveContainer" containerID="080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a"
May 29 18:25:19.021653 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.021635    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a"} err="failed to get container status \"080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a\": rpc error: code = NotFound desc = could not find container \"080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a\": container with ID starting with 080223b42a41a0422949d50e9bab1e820e2dcf289d9e354544a92737b7842e4a not found: ID does not exist"
May 29 18:25:19.021695 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.021655    2575 scope.go:117] "RemoveContainer" containerID="b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655"
May 29 18:25:19.021863 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.021845    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655"} err="failed to get container status \"b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655\": rpc error: code = NotFound desc = could not find container \"b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655\": container with ID starting with b8d87d6b12fe78fa4026a95aa7fcfdd0424cf168429df2e6992b50f85e658655 not found: ID does not exist"
May 29 18:25:19.021931 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.021865    2575 scope.go:117] "RemoveContainer" containerID="abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623"
May 29 18:25:19.022081 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.022064    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623"} err="failed to get container status \"abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623\": rpc error: code = NotFound desc = could not find container \"abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623\": container with ID starting with abb9be6149dc3211734be6f22a8439f60c52058a5c2aa52c8be31caff0eed623 not found: ID does not exist"
May 29 18:25:19.022124 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.022083    2575 scope.go:117] "RemoveContainer" containerID="707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62"
May 29 18:25:19.022300 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.022284    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62"} err="failed to get container status \"707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62\": rpc error: code = NotFound desc = could not find container \"707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62\": container with ID starting with 707eedb0762a9117a8a47eefbf45463ad467c151473b69821f3b3d261a439d62 not found: ID does not exist"
May 29 18:25:19.022353 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.022302    2575 scope.go:117] "RemoveContainer" containerID="ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41"
May 29 18:25:19.022519 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.022500    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41"} err="failed to get container status \"ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41\": rpc error: code = NotFound desc = could not find container \"ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41\": container with ID starting with ad36b3f795588eab91424af3018d9cef9acb3b9bd296b9def1fd87bcdce5ea41 not found: ID does not exist"
May 29 18:25:19.067708 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.067668    2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-offline-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:19.067708 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.067702    2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-ui-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:19.067906 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.067715    2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-online-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:19.067906 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.067743    2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/174801ae-50d3-48a8-9fd7-1de9d580e17b-feast-data\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:19.067906 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.067754    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/174801ae-50d3-48a8-9fd7-1de9d580e17b-registry-tls\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:19.067906 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.067766    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99gkn\" (UniqueName: \"kubernetes.io/projected/174801ae-50d3-48a8-9fd7-1de9d580e17b-kube-api-access-99gkn\") on node \"ip-10-0-133-167.ec2.internal\" DevicePath \"\""
May 29 18:25:19.259671 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.259635    2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"]
May 29 18:25:19.263843 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.263817    2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-2x4wb"]
May 29 18:25:19.468328 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.468297    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" path="/var/lib/kubelet/pods/174801ae-50d3-48a8-9fd7-1de9d580e17b/volumes"
May 29 18:25:19.468920 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:19.468907    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" path="/var/lib/kubelet/pods/b0a1617f-fdae-4f8e-a72c-fbb5f118a54b/volumes"
May 29 18:25:57.326253 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:57.326222    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bt5mq_a53c081c-9695-4f01-8518-64ab567e6496/global-pull-secret-syncer/0.log"
May 29 18:25:57.507333 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:57.507305    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gcd7p_0a29d4c2-23f1-4323-997a-1f5894d38144/konnectivity-agent/0.log"
May 29 18:25:57.579118 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:25:57.579053    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-167.ec2.internal_16b760888857fdc72e7a0e00f59262e9/haproxy/0.log"
May 29 18:26:00.600796 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.600720    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0536b842-caa2-4257-b24e-ce6964483d98/alertmanager/0.log"
May 29 18:26:00.643052 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.643024    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0536b842-caa2-4257-b24e-ce6964483d98/config-reloader/0.log"
May 29 18:26:00.678601 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.678577    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0536b842-caa2-4257-b24e-ce6964483d98/kube-rbac-proxy-web/0.log"
May 29 18:26:00.716209 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.716191    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0536b842-caa2-4257-b24e-ce6964483d98/kube-rbac-proxy/0.log"
May 29 18:26:00.747295 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.747276    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0536b842-caa2-4257-b24e-ce6964483d98/kube-rbac-proxy-metric/0.log"
May 29 18:26:00.777576 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.777553    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0536b842-caa2-4257-b24e-ce6964483d98/prom-label-proxy/0.log"
May 29 18:26:00.808682 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.808662    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0536b842-caa2-4257-b24e-ce6964483d98/init-config-reloader/0.log"
May 29 18:26:00.848591 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.848566    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-69b6cbdb88-757x8_049a52c2-66f9-47e2-b13b-bb28cc65d9b3/cluster-monitoring-operator/0.log"
May 29 18:26:00.878909 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.878886    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-jxvll_f4d9b51e-62a0-446b-b98a-685f0144feef/kube-state-metrics/0.log"
May 29 18:26:00.906109 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.906082    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-jxvll_f4d9b51e-62a0-446b-b98a-685f0144feef/kube-rbac-proxy-main/0.log"
May 29 18:26:00.935368 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.935348    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-jxvll_f4d9b51e-62a0-446b-b98a-685f0144feef/kube-rbac-proxy-self/0.log"
May 29 18:26:00.972174 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:00.972144    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-746d756ffc-l9sj2_b8d54a26-2978-440c-8d64-f037eb2eff84/metrics-server/0.log"
May 29 18:26:01.013525 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.013495    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-brfsr_99c5820d-68c6-4645-9daf-4a1a62155af8/monitoring-plugin/0.log"
May 29 18:26:01.129949 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.129874    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jz8kn_a397d4a0-d64c-452d-8e21-00e0030c3db8/node-exporter/0.log"
May 29 18:26:01.155749 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.155725    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jz8kn_a397d4a0-d64c-452d-8e21-00e0030c3db8/kube-rbac-proxy/0.log"
May 29 18:26:01.179473 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.179448    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jz8kn_a397d4a0-d64c-452d-8e21-00e0030c3db8/init-textfile/0.log"
May 29 18:26:01.387006 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.386924    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8f4871-47c8-476a-a5fd-1155aad26670/prometheus/0.log"
May 29 18:26:01.406648 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.406623    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8f4871-47c8-476a-a5fd-1155aad26670/config-reloader/0.log"
May 29 18:26:01.427889 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.427866    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8f4871-47c8-476a-a5fd-1155aad26670/thanos-sidecar/0.log"
May 29 18:26:01.452515 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.452494    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8f4871-47c8-476a-a5fd-1155aad26670/kube-rbac-proxy-web/0.log"
May 29 18:26:01.478905 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.478885    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8f4871-47c8-476a-a5fd-1155aad26670/kube-rbac-proxy/0.log"
May 29 18:26:01.504562 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.504539    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8f4871-47c8-476a-a5fd-1155aad26670/kube-rbac-proxy-thanos/0.log"
May 29 18:26:01.533351 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.533334    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8f4871-47c8-476a-a5fd-1155aad26670/init-config-reloader/0.log"
May 29 18:26:01.576021 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.575993    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-hfjhb_7f098669-c7a3-46ba-9cf6-421589a97e8d/prometheus-operator/0.log"
May 29 18:26:01.599733 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.599706    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-hfjhb_7f098669-c7a3-46ba-9cf6-421589a97e8d/kube-rbac-proxy/0.log"
May 29 18:26:01.631365 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.631340    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-59b7f8756b-v8flx_7a2cc0b6-442f-4e15-9f9f-1b82c6cacf8f/prometheus-operator-admission-webhook/0.log"
May 29 18:26:01.674622 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.674548    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cb66f89fc-scjh2_820f9f0a-a2f7-4418-b28c-ec862a52fe68/telemeter-client/0.log"
May 29 18:26:01.705282 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.705261    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cb66f89fc-scjh2_820f9f0a-a2f7-4418-b28c-ec862a52fe68/reload/0.log"
May 29 18:26:01.732500 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.732474    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cb66f89fc-scjh2_820f9f0a-a2f7-4418-b28c-ec862a52fe68/kube-rbac-proxy/0.log"
May 29 18:26:01.765808 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.765788    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648d5c57bb-l9d6d_9a1ee7c9-d367-4af4-8801-f4e708e62eed/thanos-query/0.log"
May 29 18:26:01.793579 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.793546    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648d5c57bb-l9d6d_9a1ee7c9-d367-4af4-8801-f4e708e62eed/kube-rbac-proxy-web/0.log"
May 29 18:26:01.819067 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.819045    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648d5c57bb-l9d6d_9a1ee7c9-d367-4af4-8801-f4e708e62eed/kube-rbac-proxy/0.log"
May 29 18:26:01.845118 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.845091    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648d5c57bb-l9d6d_9a1ee7c9-d367-4af4-8801-f4e708e62eed/prom-label-proxy/0.log"
May 29 18:26:01.866661 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.866635    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648d5c57bb-l9d6d_9a1ee7c9-d367-4af4-8801-f4e708e62eed/kube-rbac-proxy-rules/0.log"
May 29 18:26:01.887644 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:01.887621    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-648d5c57bb-l9d6d_9a1ee7c9-d367-4af4-8801-f4e708e62eed/kube-rbac-proxy-metrics/0.log"
May 29 18:26:03.545894 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.545855    2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"]
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546198    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="online"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546211    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="online"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546220    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="offline"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546226    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="offline"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546237    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="feast-init"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546243    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="feast-init"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546251    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="feast-apply"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546256    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="feast-apply"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546265    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="online"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546270    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="online"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546292    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="ui"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546297    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="ui"
May 29 18:26:03.546301 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546307    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="ui"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546312    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="ui"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546318    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="offline"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546323    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="offline"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546331    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="feast-apply"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546335    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="feast-apply"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546344    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="registry"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546349    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="registry"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546355    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="feast-init"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546360    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="feast-init"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546404    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="ui"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546413    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="online"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546421    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="registry"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546427    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="ui"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546434    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="online"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546440    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="174801ae-50d3-48a8-9fd7-1de9d580e17b" containerName="offline"
May 29 18:26:03.546697 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.546448    2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="b0a1617f-fdae-4f8e-a72c-fbb5f118a54b" containerName="offline"
May 29 18:26:03.554915 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.554883    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.557066 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.557042    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"]
May 29 18:26:03.557847 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.557829    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d5xxd\"/\"kube-root-ca.crt\""
May 29 18:26:03.557949 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.557865    2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d5xxd\"/\"openshift-service-ca.crt\""
May 29 18:26:03.559115 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.559094    2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d5xxd\"/\"default-dockercfg-gn8mp\""
May 29 18:26:03.649721 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.649696    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-proc\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.649864 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.649742    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-lib-modules\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.649864 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.649821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-podres\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.649864 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.649856    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92r52\" (UniqueName: \"kubernetes.io/projected/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-kube-api-access-92r52\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.649973 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.649893    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-sys\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.750605 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.750574    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-lib-modules\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.750763 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.750631    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-podres\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.750763 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.750749    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-92r52\" (UniqueName: \"kubernetes.io/projected/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-kube-api-access-92r52\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.750842 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.750761    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-podres\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.750842 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.750766    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-lib-modules\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.750842 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.750829    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-sys\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.750938 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.750858    2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-proc\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.750972 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.750945    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-proc\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.750972 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.750959    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-sys\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.760539 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.760517    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92r52\" (UniqueName: \"kubernetes.io/projected/7dbaabef-01cb-4ca0-baa2-8edcbadf20a9-kube-api-access-92r52\") pod \"perf-node-gather-daemonset-9694x\" (UID: \"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9\") " pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.865912 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.865883    2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:03.991811 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.991782    2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"]
May 29 18:26:03.993951 ip-10-0-133-167 kubenswrapper[2575]: W0529 18:26:03.993921    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7dbaabef_01cb_4ca0_baa2_8edcbadf20a9.slice/crio-9c89ff4c9f0ffa9b18aa166c0915a551ffd805a7a778ba34e28de2a172f9b5c5 WatchSource:0}: Error finding container 9c89ff4c9f0ffa9b18aa166c0915a551ffd805a7a778ba34e28de2a172f9b5c5: Status 404 returned error can't find the container with id 9c89ff4c9f0ffa9b18aa166c0915a551ffd805a7a778ba34e28de2a172f9b5c5
May 29 18:26:03.995460 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:03.995443    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 18:26:04.079107 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:04.079078    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x" event={"ID":"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9","Type":"ContainerStarted","Data":"3d40ff062e0c837c57979f9254d65cf71255ea43b788470985526b72d99808ae"}
May 29 18:26:04.079107 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:04.079110    2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x" event={"ID":"7dbaabef-01cb-4ca0-baa2-8edcbadf20a9","Type":"ContainerStarted","Data":"9c89ff4c9f0ffa9b18aa166c0915a551ffd805a7a778ba34e28de2a172f9b5c5"}
May 29 18:26:04.079284 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:04.079186    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:04.095848 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:04.095809    2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x" podStartSLOduration=1.095795373 podStartE2EDuration="1.095795373s" podCreationTimestamp="2026-05-29 18:26:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 18:26:04.09432776 +0000 UTC m=+1739.150121122" watchObservedRunningTime="2026-05-29 18:26:04.095795373 +0000 UTC m=+1739.151588736"
May 29 18:26:04.745113 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:04.745079    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nl6fn_eafe36c4-8553-4f04-be68-90e8803d6b0e/dns/0.log"
May 29 18:26:04.765814 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:04.765789    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nl6fn_eafe36c4-8553-4f04-be68-90e8803d6b0e/kube-rbac-proxy/0.log"
May 29 18:26:04.877041 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:04.877016    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rq5ft_2b3a52ac-9cde-4e35-b89c-f5ff5b50093c/dns-node-resolver/0.log"
May 29 18:26:05.294522 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:05.294492    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4spp5_b4578642-65b0-478f-8f73-afc1f47c3009/node-ca/0.log"
May 29 18:26:06.040692 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:06.040667    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b5cc84bff-6r2np_c74f15b7-365c-4862-bbd3-ad53e42c1d36/router/0.log"
May 29 18:26:06.368405 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:06.368378    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c547r_2c416805-3830-4517-ba34-2a7e9ec3eb8f/serve-healthcheck-canary/0.log"
May 29 18:26:06.840265 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:06.840185    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-khp6q_2f5bed7a-be5a-4b55-aa97-e52ac73285d2/kube-rbac-proxy/0.log"
May 29 18:26:06.857981 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:06.857955    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-khp6q_2f5bed7a-be5a-4b55-aa97-e52ac73285d2/exporter/0.log"
May 29 18:26:06.881052 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:06.881027    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-khp6q_2f5bed7a-be5a-4b55-aa97-e52ac73285d2/extractor/0.log"
May 29 18:26:10.092214 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:10.092187    2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d5xxd/perf-node-gather-daemonset-9694x"
May 29 18:26:11.351097 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:11.351062    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-78pw7_3aee17dc-28c0-425e-9e7b-694f66cd6459/kube-storage-version-migrator-operator/1.log"
May 29 18:26:11.352074 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:11.352058    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-78pw7_3aee17dc-28c0-425e-9e7b-694f66cd6459/kube-storage-version-migrator-operator/0.log"
May 29 18:26:12.382704 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:12.382681    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-94hsf_56fb789d-bd9e-4a88-958d-e8216ab60549/kube-multus-additional-cni-plugins/0.log"
May 29 18:26:12.403522 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:12.403499    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-94hsf_56fb789d-bd9e-4a88-958d-e8216ab60549/egress-router-binary-copy/0.log"
May 29 18:26:12.423723 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:12.423699    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-94hsf_56fb789d-bd9e-4a88-958d-e8216ab60549/cni-plugins/0.log"
May 29 18:26:12.444093 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:12.444071    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-94hsf_56fb789d-bd9e-4a88-958d-e8216ab60549/bond-cni-plugin/0.log"
May 29 18:26:12.469421 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:12.469399    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-94hsf_56fb789d-bd9e-4a88-958d-e8216ab60549/routeoverride-cni/0.log"
May 29 18:26:12.491337 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:12.491316    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-94hsf_56fb789d-bd9e-4a88-958d-e8216ab60549/whereabouts-cni-bincopy/0.log"
May 29 18:26:12.514719 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:12.514704    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-94hsf_56fb789d-bd9e-4a88-958d-e8216ab60549/whereabouts-cni/0.log"
May 29 18:26:12.977494 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:12.977411    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zctkf_c21facb6-ed73-479a-93b2-19822a7c2620/kube-multus/0.log"
May 29 18:26:13.105129 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:13.105093    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zsvdq_19a8c87f-2157-4411-8e5a-b272ce1259f5/network-metrics-daemon/0.log"
May 29 18:26:13.127188 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:13.125602    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zsvdq_19a8c87f-2157-4411-8e5a-b272ce1259f5/kube-rbac-proxy/0.log"
May 29 18:26:13.820788 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:13.820726    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-controller/0.log"
May 29 18:26:13.838879 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:13.838854    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/0.log"
May 29 18:26:13.846128 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:13.846101    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovn-acl-logging/1.log"
May 29 18:26:13.866769 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:13.866733    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/kube-rbac-proxy-node/0.log"
May 29 18:26:13.893202 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:13.893181    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/kube-rbac-proxy-ovn-metrics/0.log"
May 29 18:26:13.911774 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:13.911751    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/northd/0.log"
May 29 18:26:13.933642 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:13.933622    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/nbdb/0.log"
May 29 18:26:13.952658 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:13.952625    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/sbdb/0.log"
May 29 18:26:14.052079 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:14.052051    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cxqf9_f9ca71bf-3069-42a6-95d4-106519e08945/ovnkube-controller/0.log"
May 29 18:26:15.556476 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:15.556448    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-cp2gr_50d7a3bd-52e4-4d5d-8d79-fe64008680ab/network-check-target-container/0.log"
May 29 18:26:16.406057 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:16.406030    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-6jcqq_2eeb97a9-d522-47bc-ba71-c4fc2fc9a4eb/iptables-alerter/0.log"
May 29 18:26:17.040070 ip-10-0-133-167 kubenswrapper[2575]: I0529 18:26:17.040046    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-28z92_a054de51-319b-4ab1-b4ef-5f2d979ef630/tuned/0.log"