Jul 16 19:58:27.476535 ip-10-0-140-236 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 16 19:58:27.476547 ip-10-0-140-236 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 16 19:58:27.476556 ip-10-0-140-236 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 16 19:58:27.476863 ip-10-0-140-236 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 16 19:58:37.566787 ip-10-0-140-236 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 16 19:58:37.566803 ip-10-0-140-236 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot eef852693469490fb2cea5600a7324c6 --
Jul 16 20:00:34.467197 ip-10-0-140-236 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 20:00:34.914145 ip-10-0-140-236 systemd[1]: Started Kubernetes Kubelet.
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.842936    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848328    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848340    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848344    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848347    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848350    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848353    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848356    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:00:34.921232 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848359    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848362    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848365    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848368    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848371    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848374    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848376    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848379    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848381    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848384    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848387    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848391    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848394    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848399    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848404    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848408    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848413    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848417    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848420    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:00:34.922630 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848422    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848425    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848429    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848432    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848435    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848437    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848442    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848453    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848457    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848460    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848462    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848465    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848468    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848471    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848475    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848478    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848481    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848483    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848486    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848488    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:00:34.923624 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848491    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848493    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848496    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848499    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848501    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848504    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848507    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848509    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848512    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848514    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848517    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848519    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848522    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848525    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848528    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848532    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848536    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848539    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848542    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848546    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:00:34.924268 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848549    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848552    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848555    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848558    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848561    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848565    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848567    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848570    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848573    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848575    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848578    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848580    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848583    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848585    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848588    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848590    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848593    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848595    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848598    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848601    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:00:34.924947 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848605    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848607    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848610    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848613    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848615    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848618    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848620    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848623    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848625    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848628    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848631    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848634    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.848637    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849404    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849412    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849416    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849419    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849422    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849425    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849428    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:00:34.925718 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849431    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849434    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849436    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849439    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849442    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849445    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849449    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849453    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849456    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849459    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849462    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849465    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849468    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849471    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849473    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849476    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849478    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849481    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849483    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:00:34.926534 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849486    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849488    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849491    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849493    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849497    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849500    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849503    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849506    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849508    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849511    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849514    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849517    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849520    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849523    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849525    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849528    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849530    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849533    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849536    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849538    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:00:34.927271 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849541    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849544    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849546    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849549    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849552    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849554    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849557    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849559    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849563    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849567    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849570    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849574    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849577    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849579    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849582    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849584    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849587    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849590    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849592    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:00:34.928306 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849595    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849599    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849603    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849606    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849608    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849611    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849614    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849616    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849619    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849621    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849624    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849627    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849631    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849633    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849636    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849639    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849641    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849644    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849646    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849649    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:00:34.929336 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849652    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849654    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849657    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849660    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849662    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849665    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849668    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849671    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849673    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849676    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849679    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849681    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849685    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.849687    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850095    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850101    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850104    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850106    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850115    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850118    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:00:34.930158 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850120    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850123    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850126    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850129    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850132    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850134    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850137    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850140    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850142    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850145    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850150    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850155    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850157    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850160    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850163    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850165    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850168    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850171    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850173    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850176    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:00:34.930773 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850178    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850181    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850184    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850186    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850189    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850192    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850195    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850199    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850202    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850205    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850207    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850216    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850219    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850221    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850224    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850226    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850229    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850232    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850235    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:00:34.931407 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850237    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850239    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850242    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850246    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850251    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850253    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850256    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850259    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850261    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850264    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850268    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850272    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850275    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850278    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850281    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850284    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850286    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850289    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850291    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:00:34.932227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850294    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850296    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850299    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850301    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850304    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850308    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850317    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850319    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850322    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850324    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850327    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850331    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850334    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850337    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850340    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850343    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850348    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850352    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850355    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850357    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:00:34.932838 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850360    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850362    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850365    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850367    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850370    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850373    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850375    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850378    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850381    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850383    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850386    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850388    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850391    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850393    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.850396    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850465    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850471    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850483    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850488    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850500    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850503    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 20:00:34.933798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850507    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850511    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850514    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850526    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850530    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850533    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850536    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850541    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850545    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850548    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850551    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850554    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850560    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850563    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850566    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850570    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850573    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850577    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850580    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850583    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850586    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850589    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850592    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850595    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 20:00:34.934648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850597    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850600    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850604    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850607    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850610    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850613    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850616    2579 flags.go:64] FLAG: --enable-server="true"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850625    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850632    2579 flags.go:64] FLAG: --event-burst="100"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850635    2579 flags.go:64] FLAG: --event-qps="50"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850638    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850641    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850644    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850647    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850650    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850654    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850658    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850661    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850664    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850667    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850670    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850673    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850676    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850679    2579 flags.go:64] FLAG: --feature-gates=""
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850682    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 20:00:35.084285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850685    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850688    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850692    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850695    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850698    2579 flags.go:64] FLAG: --help="false"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850701    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-140-236.ec2.internal"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850704    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850707    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850710    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850713    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850717    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850719    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850722    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850725    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850728    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850738    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850741    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850744    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850747    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850750    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850753    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850756    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850759    2579 flags.go:64] FLAG: --lock-file=""
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850763    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 20:00:35.100426 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850766    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850769    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850775    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850777    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850780    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850783    2579 flags.go:64] FLAG: --logging-format="text"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850786    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850789    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850792    2579 flags.go:64] FLAG: --manifest-url=""
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850795    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850799    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850802    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850806    2579 flags.go:64] FLAG: --max-pods="110"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850809    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850812    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850814    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850817    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850821    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850823    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850826    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850833    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850836    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850839    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 20:00:35.101289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850842    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850851    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850857    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850860    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850863    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850866    2579 flags.go:64] FLAG: --port="10250"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850869    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850873    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b50243e88a126bed"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850876    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850880    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850883    2579 flags.go:64] FLAG: --register-node="true"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850886    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850894    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850896    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850899    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850902    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850906    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850925    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850929    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850932    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850935    2579 flags.go:64] FLAG: --runonce="false"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850937    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850940    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850943    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850946    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 20:00:35.102073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850949    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850952    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850955    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850958    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850961    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850965    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850968    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850971    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850974    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850983    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850986    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850992    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850995    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.850997    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.851005    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.851007    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.851011    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.851015    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.851018    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.851020    2579 flags.go:64] FLAG: --v="2"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.851024    2579 flags.go:64] FLAG: --version="false"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.851028    2579 flags.go:64] FLAG: --vmodule=""
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.851032    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.851035    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851166    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:00:35.103186 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851170    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851173    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851175    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851178    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851180    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851183    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851185    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851188    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851191    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851195    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851198    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851201    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851204    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851208    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851211    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851214    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851217    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851227    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851230    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:00:35.104022 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851233    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851236    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851241    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851243    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851247    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851250    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851252    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851255    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851258    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851260    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851263    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851266    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851269    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851272    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851274    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851277    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851279    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851282    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851285    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851287    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:00:35.104840 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851290    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851292    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851295    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851297    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851300    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851302    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851305    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851307    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851310    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851312    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851315    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851318    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851320    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851323    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851326    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851329    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851333    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851335    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851338    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851341    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:00:35.105645 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851343    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851346    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851349    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851351    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851354    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851357    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851360    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851362    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851365    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851367    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851370    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851373    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851375    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851379    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851382    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851385    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851388    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851390    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851393    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851395    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:00:35.106246 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851398    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851400    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851403    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851406    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851409    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851411    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851416    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851418    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851422    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851425    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851428    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851432    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851436    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851440    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851444    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851447    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851451    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851455    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:00:35.106826 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.851459    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.852642    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.859235    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.859248    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.859266    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.859271    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859336    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859341    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859345    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859348    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859351    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859353    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859356    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859359    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:00:35.107325 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859362    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859364    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859367    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859369    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859378    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859381    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859388    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859391    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859393    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859396    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859399    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859402    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859405    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859408    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859410    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859413    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859416    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859419    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859422    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859424    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:00:35.107752 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859427    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859430    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859432    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859435    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859439    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859443    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859446    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859449    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859451    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859454    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859457    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859459    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859462    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859465    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859468    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859470    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859473    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859481    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859484    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:00:35.108293 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859487    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859491    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859495    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859498    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859500    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859504    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859506    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859509    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859512    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859515    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859517    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859520    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859522    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859526    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859530    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859533    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859535    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859538    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859541    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:00:35.108844 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859543    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859546    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859548    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859550    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859553    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859556    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859558    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859561    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859563    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859566    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859569    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859571    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859579    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859581    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859584    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859587    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859589    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859592    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859594    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859597    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:00:35.109383 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859599    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859603    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859605    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859608    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859610    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859613    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859616    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859619    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859621    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859624    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859626    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859629    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859631    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.859637    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859784    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:00:35.109959 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859789    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859792    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859795    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859798    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859801    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859804    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859806    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859809    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859812    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859814    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859822    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859825    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859828    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859832    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859835    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859838    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859841    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859844    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859847    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:00:35.110372 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859852    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859854    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859857    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859860    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859862    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859865    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859868    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859870    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859873    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859875    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859878    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859880    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859883    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859886    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859888    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859891    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859893    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859896    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859898    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859902    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:00:35.110936 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859905    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859923    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859926    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859928    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859937    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859940    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859942    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859945    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859947    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859950    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859952    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859955    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859959    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859961    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859964    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859968    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859971    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859974    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859983    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:00:35.111453 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859986    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859988    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859991    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859994    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859997    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.859999    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860002    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860005    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860008    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860010    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860013    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860016    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860020    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860023    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860026    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860029    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860032    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860035    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860044    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860047    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:00:35.112023 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860050    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860052    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860055    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860057    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860060    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860064    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860066    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860069    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860071    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860074    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860077    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860079    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860082    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860084    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860087    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860091    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860094    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860097    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860100    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:00:35.112930 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:34.860102    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.860107    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.860227    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.862460    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.863637    2579 server.go:1019] "Starting client certificate rotation"
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.863743    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.863795    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.866764    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.874038    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.881024    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.888422    2579 log.go:25] "Validated CRI v1 image API"
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.889851    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 20:00:35.113614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.892267    2579 fs.go:135] Filesystem UUIDs: map[26badf57-9526-4ccc-bc5b-781acc06226b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 910b800f-c58f-46e3-9292-a15c7e544512:/dev/nvme0n1p3]
Jul 16 20:00:36.506335 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.892280    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 20:00:36.506335 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.893461    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.897374    2579 manager.go:216] Machine: {Timestamp:2026-07-16 20:00:34.896151276 +0000 UTC m=+0.317766768 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102594 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22aeb01c62ffe82e8afe17a4c36504 SystemUUID:ec22aeb0-1c62-ffe8-2e8a-fe17a4c36504 BootID:eef85269-3469-490f-b2ce-a5600a7324c6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:30:e1:f7:40:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:30:e1:f7:40:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:a8:8e:a4:a0:1e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.897474    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.897544    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.898527    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.898552    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-236.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.899738    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.899752    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.899761    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.900365    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.901593    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.904254    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.904373    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.904400    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.904411    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.904424    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.904432    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.904463    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 20:00:36.507135 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.904779    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.904794    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.907338    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.908639    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.908671    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909303    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909316    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909322    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909327    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909336    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909342    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909350    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909355    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909364    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909370    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909383    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909389    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.909992    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.910002    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.913395    2579 server.go:1268] "Started kubelet"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.913740    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.915194    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.921470    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-45kp8"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.921970    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.922049    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.922194    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-236.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:34.922555    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:34.922607    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.926083    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-45kp8"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:34.927867    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:34.927789    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-236.ec2.internal.18c2dd951de25886  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-236.ec2.internal,UID:ip-10-0-140-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-236.ec2.internal,},FirstTimestamp:2026-07-16 20:00:34.91337639 +0000 UTC m=+0.334991880,LastTimestamp:2026-07-16 20:00:34.91337639 +0000 UTC m=+0.334991880,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-236.ec2.internal,}"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.930012    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.933971    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.933978    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:34.934188    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.934933    2579 factory.go:55] Registering systemd factory
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.934973    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.934996    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.935016    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.935034    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 20:00:36.508144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.935109    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.935118    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.935209    2579 factory.go:153] Registering CRI-O factory
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.935225    2579 factory.go:223] Registration of the crio container factory successfully
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.935273    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.935297    2579 factory.go:103] Registering Raw factory
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.935314    2579 manager.go:1196] Started watching for new ooms in manager
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.936287    2579 manager.go:349] Starting recovery of all containers
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.944603    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:34.951356    2579 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-236.ec2.internal\" not found" node="ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.951397    2579 manager.go:354] Recovery completed
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.955877    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.957586    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.957610    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.957621    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.958089    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.958098    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.958097    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.958151    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.963305    2579 policy_none.go:49] "None policy: Start"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.963318    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.963329    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.963576    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.963601    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.963612    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:34.964282    2579 policy_none.go:47] "Start"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.011063    2579 manager.go:337] "Starting Device Plugin manager"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:35.011249    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.011263    2579 server.go:87] "Starting device plugin registration server"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.011553    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.011842    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.012060    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.012140    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.012149    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:35.012748    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:35.012827    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.112466    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.113590    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.113620    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.113630    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.113656    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.121644    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:35.121663    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-236.ec2.internal\": node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.122563    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.124034    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.124063    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.124090    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:35.124121    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:35.126293    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:00:36.509390 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:35.165829    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.498472    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal"]
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.498546    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.499457    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.499482    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.499493    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.500854    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.500989    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.501018    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.501657    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.501670    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.501690    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.501695    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.501702    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.501706    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.503006    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.503034    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.507885    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.507914    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.507929    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.509680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ef5df63ddb7b5d72473f9f24434be4d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"8ef5df63ddb7b5d72473f9f24434be4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.509701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ef5df63ddb7b5d72473f9f24434be4d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"8ef5df63ddb7b5d72473f9f24434be4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.510754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.509718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2bd58dc38956b1c8ecf2cf823b81fbf-config\") pod \"kube-apiserver-proxy-ip-10-0-140-236.ec2.internal\" (UID: \"b2bd58dc38956b1c8ecf2cf823b81fbf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.613011 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:36.519932    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-236.ec2.internal\" not found" node="ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.613011 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:36.525704    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-236.ec2.internal\" not found" node="ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.613011 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:36.540014    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:36.613011 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.610090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ef5df63ddb7b5d72473f9f24434be4d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"8ef5df63ddb7b5d72473f9f24434be4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.613011 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.610123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ef5df63ddb7b5d72473f9f24434be4d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"8ef5df63ddb7b5d72473f9f24434be4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.613011 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.610140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2bd58dc38956b1c8ecf2cf823b81fbf-config\") pod \"kube-apiserver-proxy-ip-10-0-140-236.ec2.internal\" (UID: \"b2bd58dc38956b1c8ecf2cf823b81fbf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.613011 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.610166    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2bd58dc38956b1c8ecf2cf823b81fbf-config\") pod \"kube-apiserver-proxy-ip-10-0-140-236.ec2.internal\" (UID: \"b2bd58dc38956b1c8ecf2cf823b81fbf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.613011 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.610176    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ef5df63ddb7b5d72473f9f24434be4d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"8ef5df63ddb7b5d72473f9f24434be4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.613011 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.610189    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ef5df63ddb7b5d72473f9f24434be4d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"8ef5df63ddb7b5d72473f9f24434be4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.728184 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:36.640188    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:36.862552 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:36.746993    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:36.862552 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.822331    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.862552 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:36.829415    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:36.862552 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:36.847922    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:37.001471 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:36.948427    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:37.135769 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:37.049407    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:37.136669 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.136536    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 20:00:37.136833 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.136671    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 20:00:37.136833 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.136675    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 20:00:37.298036 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:37.150814    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:37.298036 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.207544    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 19:55:36 +0000 UTC" deadline="2027-12-20 15:37:37.450160601 +0000 UTC"
Jul 16 20:00:37.298036 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.207566    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12523h37m0.242597241s"
Jul 16 20:00:37.298036 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:37.251267    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:37.453983 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:37.351862    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:37.453983 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:37.452158    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:37.633232 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:37.552713    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:37.633232 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.628628    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:00:37.762168 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:37.653600    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found"
Jul 16 20:00:37.762168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.706235    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:00:37.762168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.708000    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:37.762168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.721022    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 20:00:37.762168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.721850    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal"
Jul 16 20:00:37.762168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.728751    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 20:00:37.920155 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:37.859876    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef5df63ddb7b5d72473f9f24434be4d.slice/crio-1a0a450a2bc60bd2c75130e847ae3c1d2aa18febf0c0a2e2f0482771bc7457f1 WatchSource:0}: Error finding container 1a0a450a2bc60bd2c75130e847ae3c1d2aa18febf0c0a2e2f0482771bc7457f1: Status 404 returned error can't find the container with id 1a0a450a2bc60bd2c75130e847ae3c1d2aa18febf0c0a2e2f0482771bc7457f1
Jul 16 20:00:37.920155 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:37.867742    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.179150    2579 apiserver.go:52] "Watching apiserver"
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.187672    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.188027    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-j665j","kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal","openshift-dns/node-resolver-hc42k","openshift-image-registry/node-ca-5jv2r","openshift-multus/multus-8j9vc","openshift-multus/multus-additional-cni-plugins-sgkj7","openshift-multus/network-metrics-daemon-spsm5","openshift-network-diagnostics/network-check-target-4vxqj","kube-system/konnectivity-agent-cs26l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw","openshift-cluster-node-tuning-operator/tuned-tkq6d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal","openshift-network-operator/iptables-alerter-x7psl","openshift-ovn-kubernetes/ovnkube-node-wtvk2"]
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.190154    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.190221    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.191621    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.203335    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.203396    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v9cdl\""
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.203337    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.203476    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 20:00:38.207041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.206540    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.209046 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.207995    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 20:00:38.209046 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.208007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.209837 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.209823    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.210920 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.210903    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 20:00:38.211009 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.210960    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wcg4k\""
Jul 16 20:00:38.211192 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.211178    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8b2kh\""
Jul 16 20:00:38.211831 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.211814    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 20:00:38.212050 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.212028    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 20:00:38.212406 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.212392    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 20:00:38.212498 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.212483    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 20:00:38.216625 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.216608    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.217749 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.217733    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:00:38.217809 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.217797    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 20:00:38.218126 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218108    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 20:00:38.218366 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218350    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 20:00:38.218460 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218447    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4smtm\""
Jul 16 20:00:38.218573 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218536    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-var-lib-kubelet\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218573 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218553    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 20:00:38.218573 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-hostroot\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218761 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218585    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e306eeec-10d8-4f48-9456-eb6f14e7cb62-hosts-file\") pod \"node-resolver-hc42k\" (UID: \"e306eeec-10d8-4f48-9456-eb6f14e7cb62\") " pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.218761 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/906b4ac7-3b5b-4fdd-8862-ded58ddeb564-serviceca\") pod \"node-ca-5jv2r\" (UID: \"906b4ac7-3b5b-4fdd-8862-ded58ddeb564\") " pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.218761 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218637    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-os-release\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218761 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218667    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d66f5779-9348-40b9-a645-62932cbc4be6-cni-binary-copy\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218761 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218714    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-run-netns\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218761 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-var-lib-cni-multus\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218777    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 20:00:38.218999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218788    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d66f5779-9348-40b9-a645-62932cbc4be6-multus-daemon-config\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218792    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 20:00:38.218999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218812    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9m2w\" (UniqueName: \"kubernetes.io/projected/d66f5779-9348-40b9-a645-62932cbc4be6-kube-api-access-k9m2w\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p444g\" (UniqueName: \"kubernetes.io/projected/f9a71871-83ce-47f9-8eb8-5a03094d84d3-kube-api-access-p444g\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.218999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218859    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqqxc\" (UniqueName: \"kubernetes.io/projected/9d83beee-0d38-48ba-af81-3fa4fa852b48-kube-api-access-fqqxc\") pod \"iptables-alerter-x7psl\" (UID: \"9d83beee-0d38-48ba-af81-3fa4fa852b48\") " pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.218999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-run-k8s-cni-cncf-io\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218931    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-var-lib-cni-bin\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-multus-conf-dir\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.218999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.218995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f9a71871-83ce-47f9-8eb8-5a03094d84d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219018    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/906b4ac7-3b5b-4fdd-8862-ded58ddeb564-host\") pod \"node-ca-5jv2r\" (UID: \"906b4ac7-3b5b-4fdd-8862-ded58ddeb564\") " pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219029    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219039    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.219090    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219106    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-58v9s\""
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttsdj\" (UniqueName: \"kubernetes.io/projected/3d822ce7-85c4-4d38-a768-6a8fcef6109f-kube-api-access-ttsdj\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219150    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27jl9\" (UniqueName: \"kubernetes.io/projected/e306eeec-10d8-4f48-9456-eb6f14e7cb62-kube-api-access-27jl9\") pod \"node-resolver-hc42k\" (UID: \"e306eeec-10d8-4f48-9456-eb6f14e7cb62\") " pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-multus-cni-dir\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219197    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-cnibin\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f9a71871-83ce-47f9-8eb8-5a03094d84d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d83beee-0d38-48ba-af81-3fa4fa852b48-iptables-alerter-script\") pod \"iptables-alerter-x7psl\" (UID: \"9d83beee-0d38-48ba-af81-3fa4fa852b48\") " pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d83beee-0d38-48ba-af81-3fa4fa852b48-host-slash\") pod \"iptables-alerter-x7psl\" (UID: \"9d83beee-0d38-48ba-af81-3fa4fa852b48\") " pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e306eeec-10d8-4f48-9456-eb6f14e7cb62-tmp-dir\") pod \"node-resolver-hc42k\" (UID: \"e306eeec-10d8-4f48-9456-eb6f14e7cb62\") " pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.219341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-system-cni-dir\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-multus-socket-dir-parent\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-etc-kubernetes\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219398    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9a71871-83ce-47f9-8eb8-5a03094d84d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219479    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-run-multus-certs\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-system-cni-dir\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219578    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-cnibin\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-os-release\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219579    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82m6f\" (UniqueName: \"kubernetes.io/projected/906b4ac7-3b5b-4fdd-8862-ded58ddeb564-kube-api-access-82m6f\") pod \"node-ca-5jv2r\" (UID: \"906b4ac7-3b5b-4fdd-8862-ded58ddeb564\") " pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219722    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gc7dg\""
Jul 16 20:00:38.219943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.219892    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 20:00:38.220575 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.220248    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.221680 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.221662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.222587 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.222559    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 20:00:38.222587 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.222577    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tklct\""
Jul 16 20:00:38.222824 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.222807    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 20:00:38.223151 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.223134    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.223641 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.223621    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 20:00:38.223715 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.223649    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 20:00:38.223763 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.223650    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 20:00:38.223796 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.223773    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 20:00:38.223909 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.223895    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 20:00:38.224391 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.224378    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:38.224464 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.224418    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:38.224591 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.224575    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 20:00:38.224649 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.224578    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 20:00:38.224865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.224850    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-r2jcn\""
Jul 16 20:00:38.225307 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.225292    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 20:00:38.225358 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.225293    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pp4gc\""
Jul 16 20:00:38.226001 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.225986    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:00:38.230365 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.230348    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 20:00:38.253575 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.253556    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lsmws"
Jul 16 20:00:38.254961 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.254946    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:00:38.263128 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.263110    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lsmws"
Jul 16 20:00:38.310120 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.310083    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 20:00:38.319857 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.319776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-multus-conf-dir\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.319857 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.319807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/daa5599b-be4c-48b0-8370-adbba1fe8857-ovnkube-config\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.319857 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.319828    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-multus-conf-dir\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.319857 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.319841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27jl9\" (UniqueName: \"kubernetes.io/projected/e306eeec-10d8-4f48-9456-eb6f14e7cb62-kube-api-access-27jl9\") pod \"node-resolver-hc42k\" (UID: \"e306eeec-10d8-4f48-9456-eb6f14e7cb62\") " pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.319857 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.319870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f39627a-2f22-4b29-91b8-1b871181fe77-konnectivity-ca\") pod \"konnectivity-agent-cs26l\" (UID: \"6f39627a-2f22-4b29-91b8-1b871181fe77\") " pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:00:38.320169 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.319954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-var-lib-openvswitch\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.320169 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/daa5599b-be4c-48b0-8370-adbba1fe8857-ovnkube-script-lib\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.320169 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glfl5\" (UniqueName: \"kubernetes.io/projected/e2af295f-3449-40bd-8349-46b3585544ce-kube-api-access-glfl5\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.320169 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320067    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-dbus\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:38.320169 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320090    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-run-netns\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.320169 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320114    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-run-ovn-kubernetes\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.320169 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/daa5599b-be4c-48b0-8370-adbba1fe8857-ovn-node-metrics-cert\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.320169 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e2af295f-3449-40bd-8349-46b3585544ce-etc-tuned\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-etc-kubernetes\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f39627a-2f22-4b29-91b8-1b871181fe77-agent-certs\") pod \"konnectivity-agent-cs26l\" (UID: \"6f39627a-2f22-4b29-91b8-1b871181fe77\") " pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-sys\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320273    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-etc-kubernetes\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-cnibin\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320305    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-cnibin\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320310    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-os-release\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82m6f\" (UniqueName: \"kubernetes.io/projected/906b4ac7-3b5b-4fdd-8862-ded58ddeb564-kube-api-access-82m6f\") pod \"node-ca-5jv2r\" (UID: \"906b4ac7-3b5b-4fdd-8862-ded58ddeb564\") " pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-os-release\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-sysctl-conf\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-var-lib-kubelet\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-hostroot\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e306eeec-10d8-4f48-9456-eb6f14e7cb62-hosts-file\") pod \"node-resolver-hc42k\" (UID: \"e306eeec-10d8-4f48-9456-eb6f14e7cb62\") " pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320538    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/906b4ac7-3b5b-4fdd-8862-ded58ddeb564-serviceca\") pod \"node-ca-5jv2r\" (UID: \"906b4ac7-3b5b-4fdd-8862-ded58ddeb564\") " pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.320555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320553    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-var-lib-kubelet\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320570    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-hostroot\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320581    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320594    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e306eeec-10d8-4f48-9456-eb6f14e7cb62-hosts-file\") pod \"node-resolver-hc42k\" (UID: \"e306eeec-10d8-4f48-9456-eb6f14e7cb62\") " pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-systemd\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-lib-modules\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-kubelet\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320700    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d66f5779-9348-40b9-a645-62932cbc4be6-cni-binary-copy\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-run-netns\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9m2w\" (UniqueName: \"kubernetes.io/projected/d66f5779-9348-40b9-a645-62932cbc4be6-kube-api-access-k9m2w\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320767    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqqxc\" (UniqueName: \"kubernetes.io/projected/9d83beee-0d38-48ba-af81-3fa4fa852b48-kube-api-access-fqqxc\") pod \"iptables-alerter-x7psl\" (UID: \"9d83beee-0d38-48ba-af81-3fa4fa852b48\") " pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320785    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-run-netns\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320795    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-etc-openvswitch\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320828    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-node-log\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-var-lib-kubelet\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-run-k8s-cni-cncf-io\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.320950    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-var-lib-cni-bin\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.321309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321012    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-var-lib-cni-bin\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321019    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-run-k8s-cni-cncf-io\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f9a71871-83ce-47f9-8eb8-5a03094d84d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/906b4ac7-3b5b-4fdd-8862-ded58ddeb564-host\") pod \"node-ca-5jv2r\" (UID: \"906b4ac7-3b5b-4fdd-8862-ded58ddeb564\") " pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321091    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/906b4ac7-3b5b-4fdd-8862-ded58ddeb564-serviceca\") pod \"node-ca-5jv2r\" (UID: \"906b4ac7-3b5b-4fdd-8862-ded58ddeb564\") " pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/daa5599b-be4c-48b0-8370-adbba1fe8857-env-overrides\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321130    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/906b4ac7-3b5b-4fdd-8862-ded58ddeb564-host\") pod \"node-ca-5jv2r\" (UID: \"906b4ac7-3b5b-4fdd-8862-ded58ddeb564\") " pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttsdj\" (UniqueName: \"kubernetes.io/projected/3d822ce7-85c4-4d38-a768-6a8fcef6109f-kube-api-access-ttsdj\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321260    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-run-openvswitch\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321276    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-run-ovn\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321292    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-cni-bin\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321292    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321311    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fftg\" (UniqueName: \"kubernetes.io/projected/daa5599b-be4c-48b0-8370-adbba1fe8857-kube-api-access-9fftg\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-multus-cni-dir\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-cnibin\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.322018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321364    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f9a71871-83ce-47f9-8eb8-5a03094d84d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d83beee-0d38-48ba-af81-3fa4fa852b48-iptables-alerter-script\") pod \"iptables-alerter-x7psl\" (UID: \"9d83beee-0d38-48ba-af81-3fa4fa852b48\") " pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d83beee-0d38-48ba-af81-3fa4fa852b48-host-slash\") pod \"iptables-alerter-x7psl\" (UID: \"9d83beee-0d38-48ba-af81-3fa4fa852b48\") " pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321424    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e306eeec-10d8-4f48-9456-eb6f14e7cb62-tmp-dir\") pod \"node-resolver-hc42k\" (UID: \"e306eeec-10d8-4f48-9456-eb6f14e7cb62\") " pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321420    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d66f5779-9348-40b9-a645-62932cbc4be6-cni-binary-copy\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-kubelet-config\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321471    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-cnibin\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2af295f-3449-40bd-8349-46b3585544ce-tmp\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321547    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d83beee-0d38-48ba-af81-3fa4fa852b48-host-slash\") pod \"iptables-alerter-x7psl\" (UID: \"9d83beee-0d38-48ba-af81-3fa4fa852b48\") " pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321580    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-multus-cni-dir\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321619    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-sys-fs\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q95ck\" (UniqueName: \"kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck\") pod \"network-check-target-4vxqj\" (UID: \"8a72a865-5337-4bdd-8f15-68ef40f3b6f4\") " pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f9a71871-83ce-47f9-8eb8-5a03094d84d3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-cni-netd\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-system-cni-dir\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321801    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e306eeec-10d8-4f48-9456-eb6f14e7cb62-tmp-dir\") pod \"node-resolver-hc42k\" (UID: \"e306eeec-10d8-4f48-9456-eb6f14e7cb62\") " pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.322661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-multus-socket-dir-parent\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321819    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-system-cni-dir\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9a71871-83ce-47f9-8eb8-5a03094d84d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321867    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-multus-socket-dir-parent\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321876    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-slash\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321910    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-log-socket\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321932    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-modprobe-d\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.321941    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-run-multus-certs\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-system-cni-dir\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.321981    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d83beee-0d38-48ba-af81-3fa4fa852b48-iptables-alerter-script\") pod \"iptables-alerter-x7psl\" (UID: \"9d83beee-0d38-48ba-af81-3fa4fa852b48\") " pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.322008    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs podName:3d822ce7-85c4-4d38-a768-6a8fcef6109f nodeName:}" failed. No retries permitted until 2026-07-16 20:00:38.821987742 +0000 UTC m=+2.970083162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs") pod "network-metrics-daemon-spsm5" (UID: "3d822ce7-85c4-4d38-a768-6a8fcef6109f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322019    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9a71871-83ce-47f9-8eb8-5a03094d84d3-system-cni-dir\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322023    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-socket-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-run-multus-certs\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.323184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322041    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-device-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322017    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f9a71871-83ce-47f9-8eb8-5a03094d84d3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbblr\" (UniqueName: \"kubernetes.io/projected/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-kube-api-access-tbblr\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-sysconfig\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-sysctl-d\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322143    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-host\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-registration-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-run-systemd\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322244    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-kubernetes\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9a71871-83ce-47f9-8eb8-5a03094d84d3-cni-binary-copy\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-run\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-systemd-units\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-os-release\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-var-lib-cni-multus\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322452    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-os-release\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d66f5779-9348-40b9-a645-62932cbc4be6-multus-daemon-config\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.323674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p444g\" (UniqueName: \"kubernetes.io/projected/f9a71871-83ce-47f9-8eb8-5a03094d84d3-kube-api-access-p444g\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.324207 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d66f5779-9348-40b9-a645-62932cbc4be6-host-var-lib-cni-multus\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.324207 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.322899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d66f5779-9348-40b9-a645-62932cbc4be6-multus-daemon-config\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.326506 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.326488    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 20:00:38.329569 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.329535    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqqxc\" (UniqueName: \"kubernetes.io/projected/9d83beee-0d38-48ba-af81-3fa4fa852b48-kube-api-access-fqqxc\") pod \"iptables-alerter-x7psl\" (UID: \"9d83beee-0d38-48ba-af81-3fa4fa852b48\") " pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.329569 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.329552    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9m2w\" (UniqueName: \"kubernetes.io/projected/d66f5779-9348-40b9-a645-62932cbc4be6-kube-api-access-k9m2w\") pod \"multus-8j9vc\" (UID: \"d66f5779-9348-40b9-a645-62932cbc4be6\") " pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.329569 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.329560    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27jl9\" (UniqueName: \"kubernetes.io/projected/e306eeec-10d8-4f48-9456-eb6f14e7cb62-kube-api-access-27jl9\") pod \"node-resolver-hc42k\" (UID: \"e306eeec-10d8-4f48-9456-eb6f14e7cb62\") " pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.333029 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.333005    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82m6f\" (UniqueName: \"kubernetes.io/projected/906b4ac7-3b5b-4fdd-8862-ded58ddeb564-kube-api-access-82m6f\") pod \"node-ca-5jv2r\" (UID: \"906b4ac7-3b5b-4fdd-8862-ded58ddeb564\") " pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.333316 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.333276    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttsdj\" (UniqueName: \"kubernetes.io/projected/3d822ce7-85c4-4d38-a768-6a8fcef6109f-kube-api-access-ttsdj\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:38.333543 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.333519    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p444g\" (UniqueName: \"kubernetes.io/projected/f9a71871-83ce-47f9-8eb8-5a03094d84d3-kube-api-access-p444g\") pod \"multus-additional-cni-plugins-sgkj7\" (UID: \"f9a71871-83ce-47f9-8eb8-5a03094d84d3\") " pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.386232 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.386201    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:00:38.403389 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.403332    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" event={"ID":"8ef5df63ddb7b5d72473f9f24434be4d","Type":"ContainerStarted","Data":"1a0a450a2bc60bd2c75130e847ae3c1d2aa18febf0c0a2e2f0482771bc7457f1"}
Jul 16 20:00:38.405003 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.404964    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" event={"ID":"b2bd58dc38956b1c8ecf2cf823b81fbf","Type":"ContainerStarted","Data":"88776549f6853500e78f434fc12dfd2e3ad1e3cbb0614e01f2745a70d4448add"}
Jul 16 20:00:38.423464 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-sysctl-conf\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.423610 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423591    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:38.423669 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-systemd\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.423669 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-lib-modules\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.423789 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-kubelet\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.423789 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.423683    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:38.423789 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-etc-openvswitch\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.423789 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-node-log\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.423789 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-var-lib-kubelet\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.423789 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.423761    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret podName:381ae42e-c8c0-4763-b7c1-ce7fef4d6f64 nodeName:}" failed. No retries permitted until 2026-07-16 20:00:38.923739433 +0000 UTC m=+3.071834854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret") pod "global-pull-secret-syncer-j665j" (UID: "381ae42e-c8c0-4763-b7c1-ce7fef4d6f64") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/daa5599b-be4c-48b0-8370-adbba1fe8857-env-overrides\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423598    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-sysctl-conf\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423843    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-run-openvswitch\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-systemd\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423867    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-run-ovn\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-cni-bin\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fftg\" (UniqueName: \"kubernetes.io/projected/daa5599b-be4c-48b0-8370-adbba1fe8857-kube-api-access-9fftg\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-kubelet-config\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2af295f-3449-40bd-8349-46b3585544ce-tmp\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423976    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-lib-modules\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-sys-fs\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424054    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424066    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95ck\" (UniqueName: \"kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck\") pod \"network-check-target-4vxqj\" (UID: \"8a72a865-5337-4bdd-8f15-68ef40f3b6f4\") " pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:38.424093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424091    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-cni-netd\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-slash\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-log-socket\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-modprobe-d\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-socket-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424224    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-device-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.423795    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-var-lib-kubelet\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbblr\" (UniqueName: \"kubernetes.io/projected/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-kube-api-access-tbblr\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-sysconfig\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-sysctl-d\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-host\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424372    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-run-openvswitch\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-registration-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-run-systemd\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-run-ovn\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424444    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.424840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424463    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-cni-bin\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-kubernetes\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-run\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424518    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-systemd-units\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/daa5599b-be4c-48b0-8370-adbba1fe8857-ovnkube-config\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f39627a-2f22-4b29-91b8-1b871181fe77-konnectivity-ca\") pod \"konnectivity-agent-cs26l\" (UID: \"6f39627a-2f22-4b29-91b8-1b871181fe77\") " pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-var-lib-openvswitch\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/daa5599b-be4c-48b0-8370-adbba1fe8857-ovnkube-script-lib\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424666    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glfl5\" (UniqueName: \"kubernetes.io/projected/e2af295f-3449-40bd-8349-46b3585544ce-kube-api-access-glfl5\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424695    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-kubelet-config\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-dbus\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-run-netns\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424767    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-run-ovn-kubernetes\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/daa5599b-be4c-48b0-8370-adbba1fe8857-ovn-node-metrics-cert\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e2af295f-3449-40bd-8349-46b3585544ce-etc-tuned\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f39627a-2f22-4b29-91b8-1b871181fe77-agent-certs\") pod \"konnectivity-agent-cs26l\" (UID: \"6f39627a-2f22-4b29-91b8-1b871181fe77\") " pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424863    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-sys\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.425701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424952    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-sys\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.424997    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-kubelet\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425034    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-etc-openvswitch\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-run-systemd\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425077    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425118    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-node-log\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425134    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-run\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425161    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-kubernetes\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425176    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-systemd-units\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425208    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-sys-fs\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425415    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-cni-netd\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425481    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-slash\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-log-socket\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425620    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-modprobe-d\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425711    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-socket-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425760    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-device-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.425778    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/daa5599b-be4c-48b0-8370-adbba1fe8857-ovnkube-config\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.426093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/daa5599b-be4c-48b0-8370-adbba1fe8857-env-overrides\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.426469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.426234    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f39627a-2f22-4b29-91b8-1b871181fe77-konnectivity-ca\") pod \"konnectivity-agent-cs26l\" (UID: \"6f39627a-2f22-4b29-91b8-1b871181fe77\") " pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:00:38.427309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.426308    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-var-lib-openvswitch\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.427309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.426344    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-sysconfig\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.427309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.426474    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-etc-sysctl-d\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.427309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.426527    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2af295f-3449-40bd-8349-46b3585544ce-host\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.427309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.426576    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-registration-dir\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.427309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.426616    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-run-ovn-kubernetes\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.427309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.426780    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/daa5599b-be4c-48b0-8370-adbba1fe8857-ovnkube-script-lib\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.427309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.426930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/daa5599b-be4c-48b0-8370-adbba1fe8857-host-run-netns\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.427687 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.427335    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-dbus\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:38.429766 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.429653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e2af295f-3449-40bd-8349-46b3585544ce-etc-tuned\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.430722 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.429919    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f39627a-2f22-4b29-91b8-1b871181fe77-agent-certs\") pod \"konnectivity-agent-cs26l\" (UID: \"6f39627a-2f22-4b29-91b8-1b871181fe77\") " pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:00:38.430722 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.430241    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2af295f-3449-40bd-8349-46b3585544ce-tmp\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.431639 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.431609    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/daa5599b-be4c-48b0-8370-adbba1fe8857-ovn-node-metrics-cert\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.446393 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.446344    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fftg\" (UniqueName: \"kubernetes.io/projected/daa5599b-be4c-48b0-8370-adbba1fe8857-kube-api-access-9fftg\") pod \"ovnkube-node-wtvk2\" (UID: \"daa5599b-be4c-48b0-8370-adbba1fe8857\") " pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.447990 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.447718    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:00:38.447990 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.447745    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:00:38.447990 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.447759    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q95ck for pod openshift-network-diagnostics/network-check-target-4vxqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:38.447990 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.447835    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck podName:8a72a865-5337-4bdd-8f15-68ef40f3b6f4 nodeName:}" failed. No retries permitted until 2026-07-16 20:00:38.947815453 +0000 UTC m=+3.095910861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q95ck" (UniqueName: "kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck") pod "network-check-target-4vxqj" (UID: "8a72a865-5337-4bdd-8f15-68ef40f3b6f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:38.448292 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.448136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbblr\" (UniqueName: \"kubernetes.io/projected/5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3-kube-api-access-tbblr\") pod \"aws-ebs-csi-driver-node-r6ztw\" (UID: \"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.454309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.454285    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glfl5\" (UniqueName: \"kubernetes.io/projected/e2af295f-3449-40bd-8349-46b3585544ce-kube-api-access-glfl5\") pod \"tuned-tkq6d\" (UID: \"e2af295f-3449-40bd-8349-46b3585544ce\") " pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.507289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.507256    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x7psl"
Jul 16 20:00:38.519868 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.519694    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hc42k"
Jul 16 20:00:38.520656 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:38.520628    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d83beee_0d38_48ba_af81_3fa4fa852b48.slice/crio-c45122b6f16f155dc87796c6ecd9ed2b644464a35db5c3e407be2d734848c5cf WatchSource:0}: Error finding container c45122b6f16f155dc87796c6ecd9ed2b644464a35db5c3e407be2d734848c5cf: Status 404 returned error can't find the container with id c45122b6f16f155dc87796c6ecd9ed2b644464a35db5c3e407be2d734848c5cf
Jul 16 20:00:38.530913 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:38.530886    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode306eeec_10d8_4f48_9456_eb6f14e7cb62.slice/crio-3d621f923e4c0191c3b03e1fbae804bf372d9af9d25b46cca57c529e82e133e2 WatchSource:0}: Error finding container 3d621f923e4c0191c3b03e1fbae804bf372d9af9d25b46cca57c529e82e133e2: Status 404 returned error can't find the container with id 3d621f923e4c0191c3b03e1fbae804bf372d9af9d25b46cca57c529e82e133e2
Jul 16 20:00:38.537282 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.537258    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5jv2r"
Jul 16 20:00:38.542617 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.542596    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8j9vc"
Jul 16 20:00:38.550196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.550178    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sgkj7"
Jul 16 20:00:38.554596 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:38.554557    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906b4ac7_3b5b_4fdd_8862_ded58ddeb564.slice/crio-79a49404f472a76eee3d8e357506db1e9c450ee484d73eaca374033b8afc2413 WatchSource:0}: Error finding container 79a49404f472a76eee3d8e357506db1e9c450ee484d73eaca374033b8afc2413: Status 404 returned error can't find the container with id 79a49404f472a76eee3d8e357506db1e9c450ee484d73eaca374033b8afc2413
Jul 16 20:00:38.557203 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:38.557177    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66f5779_9348_40b9_a645_62932cbc4be6.slice/crio-36a735ea4a415192bfb5a46e4885e7e887914e740f0365c205c27437e28c1452 WatchSource:0}: Error finding container 36a735ea4a415192bfb5a46e4885e7e887914e740f0365c205c27437e28c1452: Status 404 returned error can't find the container with id 36a735ea4a415192bfb5a46e4885e7e887914e740f0365c205c27437e28c1452
Jul 16 20:00:38.558494 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.558324    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:00:38.561476 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:38.561448    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9a71871_83ce_47f9_8eb8_5a03094d84d3.slice/crio-fa35dd32ecc6172a7cb6a2a597a05caf2ac80d260407a8c2c6d474ba175d3974 WatchSource:0}: Error finding container fa35dd32ecc6172a7cb6a2a597a05caf2ac80d260407a8c2c6d474ba175d3974: Status 404 returned error can't find the container with id fa35dd32ecc6172a7cb6a2a597a05caf2ac80d260407a8c2c6d474ba175d3974
Jul 16 20:00:38.564476 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.564453    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:00:38.569024 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:38.568998    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f39627a_2f22_4b29_91b8_1b871181fe77.slice/crio-f9eb5e0611c7e2bdca518180e79a718fa079a0aadb0f92e46ac7a1df7be2df37 WatchSource:0}: Error finding container f9eb5e0611c7e2bdca518180e79a718fa079a0aadb0f92e46ac7a1df7be2df37: Status 404 returned error can't find the container with id f9eb5e0611c7e2bdca518180e79a718fa079a0aadb0f92e46ac7a1df7be2df37
Jul 16 20:00:38.570813 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.570637    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw"
Jul 16 20:00:38.573788 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.573641    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tkq6d"
Jul 16 20:00:38.579041 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:00:38.579011    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa5599b_be4c_48b0_8370_adbba1fe8857.slice/crio-6143e591571d317c30d6a796449d63917bca45d1ad5e9b75de8c423a829e9794 WatchSource:0}: Error finding container 6143e591571d317c30d6a796449d63917bca45d1ad5e9b75de8c423a829e9794: Status 404 returned error can't find the container with id 6143e591571d317c30d6a796449d63917bca45d1ad5e9b75de8c423a829e9794
Jul 16 20:00:38.829023 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.828903    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:38.829174 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.829068    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:38.829174 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.829139    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs podName:3d822ce7-85c4-4d38-a768-6a8fcef6109f nodeName:}" failed. No retries permitted until 2026-07-16 20:00:39.829121253 +0000 UTC m=+3.977216657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs") pod "network-metrics-daemon-spsm5" (UID: "3d822ce7-85c4-4d38-a768-6a8fcef6109f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:38.929973 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:38.929935    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:38.930143 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.930109    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:38.930195 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:38.930180    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret podName:381ae42e-c8c0-4763-b7c1-ce7fef4d6f64 nodeName:}" failed. No retries permitted until 2026-07-16 20:00:39.930162198 +0000 UTC m=+4.078257619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret") pod "global-pull-secret-syncer-j665j" (UID: "381ae42e-c8c0-4763-b7c1-ce7fef4d6f64") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:39.030923 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.030888    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95ck\" (UniqueName: \"kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck\") pod \"network-check-target-4vxqj\" (UID: \"8a72a865-5337-4bdd-8f15-68ef40f3b6f4\") " pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:39.031103 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:39.031025    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:00:39.031103 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:39.031044    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:00:39.031103 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:39.031060    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q95ck for pod openshift-network-diagnostics/network-check-target-4vxqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:39.031266 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:39.031113    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck podName:8a72a865-5337-4bdd-8f15-68ef40f3b6f4 nodeName:}" failed. No retries permitted until 2026-07-16 20:00:40.031100034 +0000 UTC m=+4.179195449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q95ck" (UniqueName: "kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck") pod "network-check-target-4vxqj" (UID: "8a72a865-5337-4bdd-8f15-68ef40f3b6f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:39.265225 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.265126    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 19:55:38 +0000 UTC" deadline="2028-02-08 08:09:47.608949204 +0000 UTC"
Jul 16 20:00:39.265225 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.265174    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13716h9m8.343780561s"
Jul 16 20:00:39.397859 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.397825    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:39.398418 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:39.397945    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:39.408221 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.408188    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hc42k" event={"ID":"e306eeec-10d8-4f48-9456-eb6f14e7cb62","Type":"ContainerStarted","Data":"3d621f923e4c0191c3b03e1fbae804bf372d9af9d25b46cca57c529e82e133e2"}
Jul 16 20:00:39.409405 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.409367    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw" event={"ID":"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3","Type":"ContainerStarted","Data":"db5f716f91c9aa7b8101b51aee333ca281c6d16da44b42c310c08eb23ee152a3"}
Jul 16 20:00:39.410553 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.410527    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2" event={"ID":"daa5599b-be4c-48b0-8370-adbba1fe8857","Type":"ContainerStarted","Data":"6143e591571d317c30d6a796449d63917bca45d1ad5e9b75de8c423a829e9794"}
Jul 16 20:00:39.411722 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.411696    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cs26l" event={"ID":"6f39627a-2f22-4b29-91b8-1b871181fe77","Type":"ContainerStarted","Data":"f9eb5e0611c7e2bdca518180e79a718fa079a0aadb0f92e46ac7a1df7be2df37"}
Jul 16 20:00:39.412955 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.412924    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8j9vc" event={"ID":"d66f5779-9348-40b9-a645-62932cbc4be6","Type":"ContainerStarted","Data":"36a735ea4a415192bfb5a46e4885e7e887914e740f0365c205c27437e28c1452"}
Jul 16 20:00:39.414087 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.414053    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5jv2r" event={"ID":"906b4ac7-3b5b-4fdd-8862-ded58ddeb564","Type":"ContainerStarted","Data":"79a49404f472a76eee3d8e357506db1e9c450ee484d73eaca374033b8afc2413"}
Jul 16 20:00:39.415218 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.415193    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x7psl" event={"ID":"9d83beee-0d38-48ba-af81-3fa4fa852b48","Type":"ContainerStarted","Data":"c45122b6f16f155dc87796c6ecd9ed2b644464a35db5c3e407be2d734848c5cf"}
Jul 16 20:00:39.416390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.416353    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tkq6d" event={"ID":"e2af295f-3449-40bd-8349-46b3585544ce","Type":"ContainerStarted","Data":"d7efe109c55140e447a7265b511f6a698319bd22f7f3f1fc2f602d0efd4cd69a"}
Jul 16 20:00:39.417478 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.417458    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgkj7" event={"ID":"f9a71871-83ce-47f9-8eb8-5a03094d84d3","Type":"ContainerStarted","Data":"fa35dd32ecc6172a7cb6a2a597a05caf2ac80d260407a8c2c6d474ba175d3974"}
Jul 16 20:00:39.836774 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.836735    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:39.836946 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:39.836891    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:39.837005 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:39.836962    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs podName:3d822ce7-85c4-4d38-a768-6a8fcef6109f nodeName:}" failed. No retries permitted until 2026-07-16 20:00:41.836942506 +0000 UTC m=+5.985037920 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs") pod "network-metrics-daemon-spsm5" (UID: "3d822ce7-85c4-4d38-a768-6a8fcef6109f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:39.937551 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:39.937509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:39.937740 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:39.937662    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:39.937803 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:39.937743    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret podName:381ae42e-c8c0-4763-b7c1-ce7fef4d6f64 nodeName:}" failed. No retries permitted until 2026-07-16 20:00:41.937723286 +0000 UTC m=+6.085818700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret") pod "global-pull-secret-syncer-j665j" (UID: "381ae42e-c8c0-4763-b7c1-ce7fef4d6f64") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:40.038803 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:40.038770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95ck\" (UniqueName: \"kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck\") pod \"network-check-target-4vxqj\" (UID: \"8a72a865-5337-4bdd-8f15-68ef40f3b6f4\") " pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:40.039061 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:40.039043    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:00:40.039164 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:40.039072    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:00:40.039164 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:40.039085    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q95ck for pod openshift-network-diagnostics/network-check-target-4vxqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:40.039164 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:40.039142    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck podName:8a72a865-5337-4bdd-8f15-68ef40f3b6f4 nodeName:}" failed. No retries permitted until 2026-07-16 20:00:42.039123926 +0000 UTC m=+6.187219349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q95ck" (UniqueName: "kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck") pod "network-check-target-4vxqj" (UID: "8a72a865-5337-4bdd-8f15-68ef40f3b6f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:40.265964 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:40.265656    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 19:55:38 +0000 UTC" deadline="2028-01-10 12:48:30.539052307 +0000 UTC"
Jul 16 20:00:40.265964 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:40.265954    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13024h47m50.273104254s"
Jul 16 20:00:40.408856 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:40.408787    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:40.409292 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:40.408909    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:40.409292 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:40.409281    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:40.409402 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:40.409369    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:40.434645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:40.434609    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" event={"ID":"b2bd58dc38956b1c8ecf2cf823b81fbf","Type":"ContainerStarted","Data":"0aee9ce1135eae0dba3a67cecdf6fb2caaa8515a947ef18007ea88e34481b177"}
Jul 16 20:00:41.399205 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:41.398692    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:41.399205 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:41.398838    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:41.459980 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:41.459939    2579 generic.go:356] "Generic (PLEG): container finished" podID="8ef5df63ddb7b5d72473f9f24434be4d" containerID="2c5b2de7006877d04b09c81c3887d6cb68072e2857f6c542c459a727042bf378" exitCode=0
Jul 16 20:00:41.460982 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:41.460954    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" event={"ID":"8ef5df63ddb7b5d72473f9f24434be4d","Type":"ContainerDied","Data":"2c5b2de7006877d04b09c81c3887d6cb68072e2857f6c542c459a727042bf378"}
Jul 16 20:00:41.476831 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:41.475982    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" podStartSLOduration=4.475962516 podStartE2EDuration="4.475962516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:00:40.44991872 +0000 UTC m=+4.598014144" watchObservedRunningTime="2026-07-16 20:00:41.475962516 +0000 UTC m=+5.624057940"
Jul 16 20:00:41.857350 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:41.856446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:41.857350 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:41.856610    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:41.857350 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:41.856675    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs podName:3d822ce7-85c4-4d38-a768-6a8fcef6109f nodeName:}" failed. No retries permitted until 2026-07-16 20:00:45.856656893 +0000 UTC m=+10.004752295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs") pod "network-metrics-daemon-spsm5" (UID: "3d822ce7-85c4-4d38-a768-6a8fcef6109f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:41.957107 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:41.957068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:41.957280 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:41.957212    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:41.957280 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:41.957276    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret podName:381ae42e-c8c0-4763-b7c1-ce7fef4d6f64 nodeName:}" failed. No retries permitted until 2026-07-16 20:00:45.957259108 +0000 UTC m=+10.105354512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret") pod "global-pull-secret-syncer-j665j" (UID: "381ae42e-c8c0-4763-b7c1-ce7fef4d6f64") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:42.057745 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:42.057708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95ck\" (UniqueName: \"kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck\") pod \"network-check-target-4vxqj\" (UID: \"8a72a865-5337-4bdd-8f15-68ef40f3b6f4\") " pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:42.057913 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:42.057887    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:00:42.057913 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:42.057908    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:00:42.058028 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:42.057921    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q95ck for pod openshift-network-diagnostics/network-check-target-4vxqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:42.058028 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:42.057979    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck podName:8a72a865-5337-4bdd-8f15-68ef40f3b6f4 nodeName:}" failed. No retries permitted until 2026-07-16 20:00:46.057960852 +0000 UTC m=+10.206056257 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q95ck" (UniqueName: "kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck") pod "network-check-target-4vxqj" (UID: "8a72a865-5337-4bdd-8f15-68ef40f3b6f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:42.402379 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:42.402346    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:42.402562 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:42.402491    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:42.402867 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:42.402844    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:42.402971 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:42.402939    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:43.397845 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:43.397807    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:43.398384 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:43.398032    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:44.402201 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:44.401554    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:44.402201 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:44.401698    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:44.402201 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:44.402052    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:44.402201 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:44.402140    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:45.398206 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:45.398174    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:45.398390 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:45.398315    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:45.892687 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:45.892606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:45.893090 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:45.892744    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:45.893090 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:45.892802    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs podName:3d822ce7-85c4-4d38-a768-6a8fcef6109f nodeName:}" failed. No retries permitted until 2026-07-16 20:00:53.892784077 +0000 UTC m=+18.040879484 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs") pod "network-metrics-daemon-spsm5" (UID: "3d822ce7-85c4-4d38-a768-6a8fcef6109f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:45.993953 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:45.993916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:45.994155 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:45.994137    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:45.994230 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:45.994212    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret podName:381ae42e-c8c0-4763-b7c1-ce7fef4d6f64 nodeName:}" failed. No retries permitted until 2026-07-16 20:00:53.994192245 +0000 UTC m=+18.142287649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret") pod "global-pull-secret-syncer-j665j" (UID: "381ae42e-c8c0-4763-b7c1-ce7fef4d6f64") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:46.095825 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:46.095223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95ck\" (UniqueName: \"kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck\") pod \"network-check-target-4vxqj\" (UID: \"8a72a865-5337-4bdd-8f15-68ef40f3b6f4\") " pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:46.095825 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:46.095393    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:00:46.095825 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:46.095415    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:00:46.095825 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:46.095427    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q95ck for pod openshift-network-diagnostics/network-check-target-4vxqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:46.095825 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:46.095499    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck podName:8a72a865-5337-4bdd-8f15-68ef40f3b6f4 nodeName:}" failed. No retries permitted until 2026-07-16 20:00:54.095481297 +0000 UTC m=+18.243576700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q95ck" (UniqueName: "kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck") pod "network-check-target-4vxqj" (UID: "8a72a865-5337-4bdd-8f15-68ef40f3b6f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:46.399128 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:46.399102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:46.399283 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:46.399209    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:46.399364 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:46.399293    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:46.399414 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:46.399398    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:47.398051 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:47.397980    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:47.398553 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:47.398124    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:48.398553 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:48.398509    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:48.399027 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:48.398524    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:48.399027 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:48.398690    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:48.399027 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:48.398748    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:49.398973 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:49.398928    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:49.399365 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:49.399084    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:50.398661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:50.398584    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:50.398841 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:50.398726    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:50.398841 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:50.398777    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:50.398923 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:50.398889    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:51.398548 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:51.398508    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:51.398999 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:51.398653    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:52.397838 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:52.397806    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:52.398017 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:52.397985    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:52.398085 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:52.398040    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:52.398186 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:52.398165    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:53.398930 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:53.398888    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:53.399410 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:53.399053    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:53.950185 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:53.950136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:53.950394 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:53.950290    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:53.950394 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:53.950366    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs podName:3d822ce7-85c4-4d38-a768-6a8fcef6109f nodeName:}" failed. No retries permitted until 2026-07-16 20:01:09.950348177 +0000 UTC m=+34.098443581 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs") pod "network-metrics-daemon-spsm5" (UID: "3d822ce7-85c4-4d38-a768-6a8fcef6109f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:00:54.051225 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:54.051194    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:54.051411 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:54.051363    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:54.051512 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:54.051457    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret podName:381ae42e-c8c0-4763-b7c1-ce7fef4d6f64 nodeName:}" failed. No retries permitted until 2026-07-16 20:01:10.051420887 +0000 UTC m=+34.199516307 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret") pod "global-pull-secret-syncer-j665j" (UID: "381ae42e-c8c0-4763-b7c1-ce7fef4d6f64") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:00:54.152655 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:54.152618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95ck\" (UniqueName: \"kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck\") pod \"network-check-target-4vxqj\" (UID: \"8a72a865-5337-4bdd-8f15-68ef40f3b6f4\") " pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:54.152813 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:54.152792    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:00:54.152813 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:54.152813    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:00:54.152908 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:54.152827    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q95ck for pod openshift-network-diagnostics/network-check-target-4vxqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:54.152908 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:54.152891    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck podName:8a72a865-5337-4bdd-8f15-68ef40f3b6f4 nodeName:}" failed. No retries permitted until 2026-07-16 20:01:10.152871985 +0000 UTC m=+34.300967403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q95ck" (UniqueName: "kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck") pod "network-check-target-4vxqj" (UID: "8a72a865-5337-4bdd-8f15-68ef40f3b6f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:00:54.398085 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:54.398007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:54.398226 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:54.398010    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:54.398226 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:54.398147    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:54.398305 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:54.398216    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:55.398258 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:55.398226    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:55.398761 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:55.398367    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:56.398865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:56.398828    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:56.399315 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:56.398924    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:56.399315 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:56.398999    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:56.399315 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:56.399085    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:57.398468 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:57.398416    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:57.398617 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:57.398561    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:58.400509 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.400353    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:00:58.401559 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.400408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:00:58.401559 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:58.400600    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:00:58.401559 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:58.400692    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:00:58.498008 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.497786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw" event={"ID":"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3","Type":"ContainerStarted","Data":"a0feefcb1decafc672bc867d1feb0d4667930409f2d2975464557d5445cc26b2"}
Jul 16 20:00:58.499718 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.499689    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2" event={"ID":"daa5599b-be4c-48b0-8370-adbba1fe8857","Type":"ContainerStarted","Data":"a90dec455ab9c34e2a1a3bff4b40361461ca2ebbcbc633613601d862a8ee06e9"}
Jul 16 20:00:58.499866 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.499759    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2" event={"ID":"daa5599b-be4c-48b0-8370-adbba1fe8857","Type":"ContainerStarted","Data":"a45971509c871b490d543bcf42e20cdefd4bceac0d95a87faab1b9d3b05ec6a1"}
Jul 16 20:00:58.501140 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.501115    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cs26l" event={"ID":"6f39627a-2f22-4b29-91b8-1b871181fe77","Type":"ContainerStarted","Data":"5651cdac9ebec58e1af5f1414393950769ffeba217267b16b7a9fd7a618405a5"}
Jul 16 20:00:58.502546 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.502526    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8j9vc" event={"ID":"d66f5779-9348-40b9-a645-62932cbc4be6","Type":"ContainerStarted","Data":"b5c7dd0440b3abcc1d118957af8b1039a682be7e5c4f19846c9eab5c87d90cb4"}
Jul 16 20:00:58.507168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.507139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5jv2r" event={"ID":"906b4ac7-3b5b-4fdd-8862-ded58ddeb564","Type":"ContainerStarted","Data":"845fb681174c4584c0ba8f88f137d8c39b46cf15645c57cba165ad1e94ebb1f9"}
Jul 16 20:00:58.509290 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.509263    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" event={"ID":"8ef5df63ddb7b5d72473f9f24434be4d","Type":"ContainerStarted","Data":"8ff6c644b917542f3adf126ecfeb5cc1bf8d8c5c5adea93eaa418350fcd7f523"}
Jul 16 20:00:58.511097 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.511077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tkq6d" event={"ID":"e2af295f-3449-40bd-8349-46b3585544ce","Type":"ContainerStarted","Data":"e1a34a651c7e9115e0a3f8ce6dfe0018f789f2fdf157de0c1e278def497d6888"}
Jul 16 20:00:58.513038 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.513012    2579 generic.go:356] "Generic (PLEG): container finished" podID="f9a71871-83ce-47f9-8eb8-5a03094d84d3" containerID="ea1ea1163c13b0b0f76f19f0cb0a690016550b1cb1d9c6893a3c4820ffbde6da" exitCode=0
Jul 16 20:00:58.513124 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.513084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgkj7" event={"ID":"f9a71871-83ce-47f9-8eb8-5a03094d84d3","Type":"ContainerDied","Data":"ea1ea1163c13b0b0f76f19f0cb0a690016550b1cb1d9c6893a3c4820ffbde6da"}
Jul 16 20:00:58.514739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.514715    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hc42k" event={"ID":"e306eeec-10d8-4f48-9456-eb6f14e7cb62","Type":"ContainerStarted","Data":"a45f37db4233d4c4c347bacd5500d87dae5daa4afc860918053e23e18388af8f"}
Jul 16 20:00:58.519349 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.519295    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cs26l" podStartSLOduration=7.572893655 podStartE2EDuration="22.519277368s" totalImagesPullingTime="14.946383713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:00:58.518636675 +0000 UTC m=+22.666732099" watchObservedRunningTime="2026-07-16 20:00:58.519277368 +0000 UTC m=+22.667372855"
Jul 16 20:00:58.538852 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.538785    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tkq6d" podStartSLOduration=3.026736354 podStartE2EDuration="22.538770794s" totalImagesPullingTime="19.51203444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:00:58.538278147 +0000 UTC m=+22.686373591" watchObservedRunningTime="2026-07-16 20:00:58.538770794 +0000 UTC m=+22.686866217"
Jul 16 20:00:58.554933 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.554885    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" podStartSLOduration=21.554870289 podStartE2EDuration="21.554870289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:00:58.55459287 +0000 UTC m=+22.702688294" watchObservedRunningTime="2026-07-16 20:00:58.554870289 +0000 UTC m=+22.702965712"
Jul 16 20:00:58.573215 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.573171    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hc42k" podStartSLOduration=3.008181252 podStartE2EDuration="22.573157222s" totalImagesPullingTime="19.56497597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:00:58.57266627 +0000 UTC m=+22.720761696" watchObservedRunningTime="2026-07-16 20:00:58.573157222 +0000 UTC m=+22.721252644"
Jul 16 20:00:58.613861 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.613815    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5jv2r" podStartSLOduration=7.651354519 podStartE2EDuration="22.613798735s" totalImagesPullingTime="14.962444216s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:00:58.613355378 +0000 UTC m=+22.761450803" watchObservedRunningTime="2026-07-16 20:00:58.613798735 +0000 UTC m=+22.761894159"
Jul 16 20:00:58.613998 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.613887    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8j9vc" podStartSLOduration=2.925367496 podStartE2EDuration="22.613881962s" totalImagesPullingTime="19.688514466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:00:58.598053989 +0000 UTC m=+22.746149412" watchObservedRunningTime="2026-07-16 20:00:58.613881962 +0000 UTC m=+22.761977384"
Jul 16 20:00:58.916579 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:58.916511    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5jv2r_906b4ac7-3b5b-4fdd-8862-ded58ddeb564/node-ca/0.log"
Jul 16 20:00:59.398327 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:59.398299    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:00:59.398700 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:00:59.398451    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:00:59.520678 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:59.520643    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2" event={"ID":"daa5599b-be4c-48b0-8370-adbba1fe8857","Type":"ContainerStarted","Data":"28c3f5bc9f57f65eb95d874ca8f1ac40fa44e6760f2b3e2a4dac3df81f8ac3e1"}
Jul 16 20:00:59.521037 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:59.520744    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2" event={"ID":"daa5599b-be4c-48b0-8370-adbba1fe8857","Type":"ContainerStarted","Data":"0a60520c2dbf8958e842ac3169eed7a67b00fc3cbf816d743b63b4eda659f73e"}
Jul 16 20:00:59.521037 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:59.520799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2" event={"ID":"daa5599b-be4c-48b0-8370-adbba1fe8857","Type":"ContainerStarted","Data":"4a4a4153529ee4694ce775d86bf2a8d1df5e5827a79d03ed3646cded6ef910fd"}
Jul 16 20:00:59.521037 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:59.520840    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2" event={"ID":"daa5599b-be4c-48b0-8370-adbba1fe8857","Type":"ContainerStarted","Data":"f82e796b11a7a343f3876301fe7af679390f1cb683a11dc3335dc2f55e6a2a00"}
Jul 16 20:00:59.522107 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:00:59.522082    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x7psl" event={"ID":"9d83beee-0d38-48ba-af81-3fa4fa852b48","Type":"ContainerStarted","Data":"8e38f5b4d09ef370e13f414e05e610488471f5991ae94e6d7060070078395699"}
Jul 16 20:01:00.229273 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:00.229238    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 20:01:00.299524 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:00.299407    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T20:01:00.229262314Z","UUID":"d1c76926-5058-41f9-a52e-df4bf0faf080","Handler":null,"Name":"","Endpoint":""}
Jul 16 20:01:00.300927 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:00.300909    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 20:01:00.301026 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:00.300949    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 20:01:00.398116 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:00.398043    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:00.398271 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:00.398043    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:00.398271 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:00.398156    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:00.398271 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:00.398235    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:00.525326 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:00.525288    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw" event={"ID":"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3","Type":"ContainerStarted","Data":"17a9ede3daf2578b1faee27632844541436e842211ba5548accb0b45ce39c1c5"}
Jul 16 20:01:01.398945 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:01.398762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:01.399121 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:01.399056    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:01.530783 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:01.530742    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2" event={"ID":"daa5599b-be4c-48b0-8370-adbba1fe8857","Type":"ContainerStarted","Data":"a22dff57b4a82ffa5f99a782535a0746836399cd73fe186be72bd7b7f0047ec4"}
Jul 16 20:01:02.397844 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:02.397817    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:02.397924 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:02.397871    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:02.397977 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:02.397959    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:02.398036 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:02.398020    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:02.536158 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:02.536124    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw" event={"ID":"5fb6ec74-2c4f-4c85-9f4b-7d9cb80392d3","Type":"ContainerStarted","Data":"3a6df4ddf91fa4cf58c55de0f0166f1cea35c4d01ed5f144305a8eb69aae9449"}
Jul 16 20:01:02.559065 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:02.559012    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6ztw" podStartSLOduration=3.109889152 podStartE2EDuration="26.558992699s" totalImagesPullingTime="23.449103547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:02.558769839 +0000 UTC m=+26.706865262" watchObservedRunningTime="2026-07-16 20:01:02.558992699 +0000 UTC m=+26.707088123"
Jul 16 20:01:02.559303 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:02.559277    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x7psl" podStartSLOduration=6.984011486 podStartE2EDuration="26.559266611s" totalImagesPullingTime="19.575255125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:00.538235072 +0000 UTC m=+24.686330496" watchObservedRunningTime="2026-07-16 20:01:02.559266611 +0000 UTC m=+26.707362037"
Jul 16 20:01:02.606075 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:02.606037    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:01:02.606762 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:02.606739    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:01:03.399120 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:03.398815    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:03.399262 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:03.399163    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:03.543200 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:03.543155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2" event={"ID":"daa5599b-be4c-48b0-8370-adbba1fe8857","Type":"ContainerStarted","Data":"a2e3aceffea1750bb32e4cf404190f846b8e35c4ed143f1ee637852f2e35fb30"}
Jul 16 20:01:03.543969 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:03.543382    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:01:03.544129 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:03.544110    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cs26l"
Jul 16 20:01:03.544187 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:03.544148    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:01:03.561908 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:03.561882    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:01:03.574400 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:03.574355    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2" podStartSLOduration=8.004343427 podStartE2EDuration="27.574341026s" totalImagesPullingTime="19.569997599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:03.573135362 +0000 UTC m=+27.721230786" watchObservedRunningTime="2026-07-16 20:01:03.574341026 +0000 UTC m=+27.722436449"
Jul 16 20:01:04.398445 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:04.398409    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:04.398613 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:04.398454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:04.398613 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:04.398552    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:04.398723 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:04.398695    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:04.544792 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:04.544730    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:01:04.545199 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:04.544811    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:01:04.560387 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:04.560363    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:01:05.397844 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:05.397803    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:05.398013 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:05.397925    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:06.398714 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:06.398683    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:06.399203 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:06.398786    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:06.399203 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:06.398863    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:06.399203 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:06.398949    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:07.397969 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:07.397932    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:07.398131 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:07.398050    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:07.552224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:07.552187    2579 generic.go:356] "Generic (PLEG): container finished" podID="f9a71871-83ce-47f9-8eb8-5a03094d84d3" containerID="07430680618e95940194e7d2197cbc767f33b20fc7811325f09e39220cf69dca" exitCode=0
Jul 16 20:01:07.552613 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:07.552248    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgkj7" event={"ID":"f9a71871-83ce-47f9-8eb8-5a03094d84d3","Type":"ContainerDied","Data":"07430680618e95940194e7d2197cbc767f33b20fc7811325f09e39220cf69dca"}
Jul 16 20:01:08.400178 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:08.400139    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:08.400337 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:08.400288    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:08.400395 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:08.400375    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:08.400531 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:08.400509    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:09.398700 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:09.398663    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:09.399073 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:09.398786    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:09.558648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:09.558596    2579 generic.go:356] "Generic (PLEG): container finished" podID="f9a71871-83ce-47f9-8eb8-5a03094d84d3" containerID="a3e74c833dc54c457901f85776a07302016624ecf55fc6bf7f34b0c4e212d682" exitCode=0
Jul 16 20:01:09.558875 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:09.558681    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgkj7" event={"ID":"f9a71871-83ce-47f9-8eb8-5a03094d84d3","Type":"ContainerDied","Data":"a3e74c833dc54c457901f85776a07302016624ecf55fc6bf7f34b0c4e212d682"}
Jul 16 20:01:09.971424 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:09.971362    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:09.971758 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:09.971567    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:01:09.971758 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:09.971654    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs podName:3d822ce7-85c4-4d38-a768-6a8fcef6109f nodeName:}" failed. No retries permitted until 2026-07-16 20:01:41.971632178 +0000 UTC m=+66.119727582 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs") pod "network-metrics-daemon-spsm5" (UID: "3d822ce7-85c4-4d38-a768-6a8fcef6109f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:01:10.072265 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:10.072216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:10.072546 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:10.072383    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:01:10.072546 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:10.072483    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret podName:381ae42e-c8c0-4763-b7c1-ce7fef4d6f64 nodeName:}" failed. No retries permitted until 2026-07-16 20:01:42.072459475 +0000 UTC m=+66.220554878 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret") pod "global-pull-secret-syncer-j665j" (UID: "381ae42e-c8c0-4763-b7c1-ce7fef4d6f64") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:01:10.172908 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:10.172873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95ck\" (UniqueName: \"kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck\") pod \"network-check-target-4vxqj\" (UID: \"8a72a865-5337-4bdd-8f15-68ef40f3b6f4\") " pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:10.173082 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:10.173021    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:01:10.173082 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:10.173035    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:01:10.173082 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:10.173044    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q95ck for pod openshift-network-diagnostics/network-check-target-4vxqj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:01:10.173181 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:10.173091    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck podName:8a72a865-5337-4bdd-8f15-68ef40f3b6f4 nodeName:}" failed. No retries permitted until 2026-07-16 20:01:42.173079199 +0000 UTC m=+66.321174599 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q95ck" (UniqueName: "kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck") pod "network-check-target-4vxqj" (UID: "8a72a865-5337-4bdd-8f15-68ef40f3b6f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:01:10.397993 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:10.397967    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:10.398095 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:10.397968    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:10.398095 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:10.398074    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:10.398200 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:10.398178    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:10.563254 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:10.563213    2579 generic.go:356] "Generic (PLEG): container finished" podID="f9a71871-83ce-47f9-8eb8-5a03094d84d3" containerID="dde7441cf46c71c1a50aaaafb1047c31d920cfa4a1c24584f9af35d2a39e3482" exitCode=0
Jul 16 20:01:10.563639 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:10.563289    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgkj7" event={"ID":"f9a71871-83ce-47f9-8eb8-5a03094d84d3","Type":"ContainerDied","Data":"dde7441cf46c71c1a50aaaafb1047c31d920cfa4a1c24584f9af35d2a39e3482"}
Jul 16 20:01:11.398493 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:11.398456    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:11.398689 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:11.398596    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:12.398384 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:12.398353    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:12.398778 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:12.398393    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:12.398778 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:12.398484    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:12.398778 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:12.398593    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:13.397897 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:13.397851    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:13.398070 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:13.398006    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:14.398417 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:14.398377    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:14.398917 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:14.398541    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:14.398917 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:14.398593    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:14.398917 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:14.398700    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:15.398512 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:15.398486    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:15.398878 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:15.398628    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:15.577374 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:15.577289    2579 generic.go:356] "Generic (PLEG): container finished" podID="f9a71871-83ce-47f9-8eb8-5a03094d84d3" containerID="effb60008845beb18862117315bccdef33dde380e34dfed7e56efb6f02f72f6a" exitCode=0
Jul 16 20:01:15.577374 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:15.577346    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgkj7" event={"ID":"f9a71871-83ce-47f9-8eb8-5a03094d84d3","Type":"ContainerDied","Data":"effb60008845beb18862117315bccdef33dde380e34dfed7e56efb6f02f72f6a"}
Jul 16 20:01:16.399645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:16.399601    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:16.400175 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:16.399711    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:16.400175 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:16.399754    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:16.400175 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:16.399847    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:16.581832 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:16.581795    2579 generic.go:356] "Generic (PLEG): container finished" podID="f9a71871-83ce-47f9-8eb8-5a03094d84d3" containerID="cf27d22fcfb2b6ffcc504313a1b5c392578eec4b64d5407663be403355b9a211" exitCode=0
Jul 16 20:01:16.582032 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:16.581847    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgkj7" event={"ID":"f9a71871-83ce-47f9-8eb8-5a03094d84d3","Type":"ContainerDied","Data":"cf27d22fcfb2b6ffcc504313a1b5c392578eec4b64d5407663be403355b9a211"}
Jul 16 20:01:17.398040 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:17.398005    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:17.398223 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:17.398116    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:17.587273 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:17.587236    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sgkj7" event={"ID":"f9a71871-83ce-47f9-8eb8-5a03094d84d3","Type":"ContainerStarted","Data":"61f16e7d366f6e56e41b2b7af8271c5c1a82b7baa86072047f1a4e9e7273ae4f"}
Jul 16 20:01:17.614645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:17.614595    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sgkj7" podStartSLOduration=7.374314297 podStartE2EDuration="41.614580857s" totalImagesPullingTime="34.24026656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:17.614364341 +0000 UTC m=+41.762459768" watchObservedRunningTime="2026-07-16 20:01:17.614580857 +0000 UTC m=+41.762676280"
Jul 16 20:01:18.398549 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:18.398508    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:18.398549 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:18.398543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:18.398753 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:18.398624    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:18.398753 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:18.398734    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:19.398238 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:19.398199    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:19.398635 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:19.398347    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:20.398171 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:20.398134    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:20.398349 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:20.398140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:20.398349 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:20.398243    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:20.398349 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:20.398306    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:21.398250 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:21.398210    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:21.398402 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:21.398326    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:22.398142 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:22.398109    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:22.398346 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:22.398248    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:22.398454 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:22.398110    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:22.398848 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:22.398519    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:23.394004 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:23.393974    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j665j"]
Jul 16 20:01:23.394153 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:23.394073    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:23.394191 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:23.394151    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:23.398005 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:23.397980    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-spsm5"]
Jul 16 20:01:23.398130 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:23.398086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:23.398218 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:23.398196    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:23.398824 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:23.398796    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4vxqj"]
Jul 16 20:01:23.399579 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:23.398881    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:23.399579 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:23.398969    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:23.602497 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:23.602453    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:23.602667 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:23.602579    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:25.398969 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:25.398780    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:25.399410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:25.398818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:25.399410 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:25.399065    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-spsm5" podUID="3d822ce7-85c4-4d38-a768-6a8fcef6109f"
Jul 16 20:01:25.399410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:25.398818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:25.399410 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:25.399129    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j665j" podUID="381ae42e-c8c0-4763-b7c1-ce7fef4d6f64"
Jul 16 20:01:25.399410 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:25.399230    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4vxqj" podUID="8a72a865-5337-4bdd-8f15-68ef40f3b6f4"
Jul 16 20:01:27.225422 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.225395    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeReady"
Jul 16 20:01:27.225831 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.225520    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 20:01:27.295685 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.295655    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-97vtr"]
Jul 16 20:01:27.315934 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.315810    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9t7nr"]
Jul 16 20:01:27.316127 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.315987    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-97vtr"
Jul 16 20:01:27.318742 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.318718    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 20:01:27.318884 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.318720    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 20:01:27.318884 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.318805    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mtlsx\""
Jul 16 20:01:27.318884 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.318884    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 20:01:27.331827 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.331807    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-97vtr"]
Jul 16 20:01:27.331914 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.331836    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9t7nr"]
Jul 16 20:01:27.331957 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.331911    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.334880 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.334864    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 20:01:27.335048 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.335029    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 20:01:27.335256 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.335241    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-92cdv\""
Jul 16 20:01:27.335349 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.335333    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 20:01:27.335390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.335335    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 20:01:27.389189 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.389157    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lvjxn"]
Jul 16 20:01:27.400299 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.400273    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-data-volume\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.400425 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.400302    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.400425 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.400321    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbtg7\" (UniqueName: \"kubernetes.io/projected/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-kube-api-access-wbtg7\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.400425 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.400354    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.400619 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.400426    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmcrd\" (UniqueName: \"kubernetes.io/projected/de75ba51-01b2-48d4-b7f8-c0c803fa0eba-kube-api-access-zmcrd\") pod \"ingress-canary-97vtr\" (UID: \"de75ba51-01b2-48d4-b7f8-c0c803fa0eba\") " pod="openshift-ingress-canary/ingress-canary-97vtr"
Jul 16 20:01:27.400619 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.400509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de75ba51-01b2-48d4-b7f8-c0c803fa0eba-cert\") pod \"ingress-canary-97vtr\" (UID: \"de75ba51-01b2-48d4-b7f8-c0c803fa0eba\") " pod="openshift-ingress-canary/ingress-canary-97vtr"
Jul 16 20:01:27.400619 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.400560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-crio-socket\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.415772 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.415753    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lvjxn"]
Jul 16 20:01:27.415877 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.415870    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:27.415969 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.415953    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:27.416211 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.416190    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:27.416323 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.416311    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.418545 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.418524    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 20:01:27.418666 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.418606    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 20:01:27.418666 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.418620    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 20:01:27.418763 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.418753    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 20:01:27.418815 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.418791    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 20:01:27.419057 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.419040    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 20:01:27.419120 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.419080    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fkb6w\""
Jul 16 20:01:27.419168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.419154    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nxdf6\""
Jul 16 20:01:27.419226 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.419211    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sfzlm\""
Jul 16 20:01:27.501101 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501032    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-data-volume\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.501101 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.501101 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501093    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbtg7\" (UniqueName: \"kubernetes.io/projected/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-kube-api-access-wbtg7\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.501328 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.501328 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501292    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc035797-bbb8-4f53-bad4-95d700ac5ce7-metrics-tls\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.501455 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501333    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc035797-bbb8-4f53-bad4-95d700ac5ce7-config-volume\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.501455 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501347    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-data-volume\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.501455 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc035797-bbb8-4f53-bad4-95d700ac5ce7-tmp-dir\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.501455 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmcrd\" (UniqueName: \"kubernetes.io/projected/de75ba51-01b2-48d4-b7f8-c0c803fa0eba-kube-api-access-zmcrd\") pod \"ingress-canary-97vtr\" (UID: \"de75ba51-01b2-48d4-b7f8-c0c803fa0eba\") " pod="openshift-ingress-canary/ingress-canary-97vtr"
Jul 16 20:01:27.501656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de75ba51-01b2-48d4-b7f8-c0c803fa0eba-cert\") pod \"ingress-canary-97vtr\" (UID: \"de75ba51-01b2-48d4-b7f8-c0c803fa0eba\") " pod="openshift-ingress-canary/ingress-canary-97vtr"
Jul 16 20:01:27.501656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501516    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmf46\" (UniqueName: \"kubernetes.io/projected/cc035797-bbb8-4f53-bad4-95d700ac5ce7-kube-api-access-fmf46\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.501656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-crio-socket\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.501801 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501760    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-crio-socket\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.501801 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.501764    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.504995 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.504966    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.505139 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.505121    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de75ba51-01b2-48d4-b7f8-c0c803fa0eba-cert\") pod \"ingress-canary-97vtr\" (UID: \"de75ba51-01b2-48d4-b7f8-c0c803fa0eba\") " pod="openshift-ingress-canary/ingress-canary-97vtr"
Jul 16 20:01:27.509197 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.509172    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmcrd\" (UniqueName: \"kubernetes.io/projected/de75ba51-01b2-48d4-b7f8-c0c803fa0eba-kube-api-access-zmcrd\") pod \"ingress-canary-97vtr\" (UID: \"de75ba51-01b2-48d4-b7f8-c0c803fa0eba\") " pod="openshift-ingress-canary/ingress-canary-97vtr"
Jul 16 20:01:27.509532 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.509502    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbtg7\" (UniqueName: \"kubernetes.io/projected/9247a48e-28a7-4b4d-aa29-9d6d65e943d8-kube-api-access-wbtg7\") pod \"insights-runtime-extractor-9t7nr\" (UID: \"9247a48e-28a7-4b4d-aa29-9d6d65e943d8\") " pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.602736 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.602708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc035797-bbb8-4f53-bad4-95d700ac5ce7-metrics-tls\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.602736 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.602739    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc035797-bbb8-4f53-bad4-95d700ac5ce7-config-volume\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.602899 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.602757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc035797-bbb8-4f53-bad4-95d700ac5ce7-tmp-dir\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.602899 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.602810    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmf46\" (UniqueName: \"kubernetes.io/projected/cc035797-bbb8-4f53-bad4-95d700ac5ce7-kube-api-access-fmf46\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.603187 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.603169    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc035797-bbb8-4f53-bad4-95d700ac5ce7-tmp-dir\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.603246 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.603219    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc035797-bbb8-4f53-bad4-95d700ac5ce7-config-volume\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.604893 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.604877    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc035797-bbb8-4f53-bad4-95d700ac5ce7-metrics-tls\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.610927 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.610905    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmf46\" (UniqueName: \"kubernetes.io/projected/cc035797-bbb8-4f53-bad4-95d700ac5ce7-kube-api-access-fmf46\") pod \"dns-default-lvjxn\" (UID: \"cc035797-bbb8-4f53-bad4-95d700ac5ce7\") " pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.625770 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.625752    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-97vtr"
Jul 16 20:01:27.639937 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.639922    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9t7nr"
Jul 16 20:01:27.752909 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.752632    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:27.827024 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.826994    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-97vtr"]
Jul 16 20:01:27.831856 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:27.831825    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde75ba51_01b2_48d4_b7f8_c0c803fa0eba.slice/crio-55ef2c3d56016d8da1a4b29a607237ab20731acb596bca6947b39248a594ae88 WatchSource:0}: Error finding container 55ef2c3d56016d8da1a4b29a607237ab20731acb596bca6947b39248a594ae88: Status 404 returned error can't find the container with id 55ef2c3d56016d8da1a4b29a607237ab20731acb596bca6947b39248a594ae88
Jul 16 20:01:27.839633 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.839610    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9t7nr"]
Jul 16 20:01:27.842782 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:27.842753    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9247a48e_28a7_4b4d_aa29_9d6d65e943d8.slice/crio-e07b92db08fd31612b5bd29a32e974981e9e7879dbb7662828009df8113a84fb WatchSource:0}: Error finding container e07b92db08fd31612b5bd29a32e974981e9e7879dbb7662828009df8113a84fb: Status 404 returned error can't find the container with id e07b92db08fd31612b5bd29a32e974981e9e7879dbb7662828009df8113a84fb
Jul 16 20:01:27.898453 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:27.898404    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lvjxn"]
Jul 16 20:01:27.901307 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:27.901282    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc035797_bbb8_4f53_bad4_95d700ac5ce7.slice/crio-073ad7b07bc0b49016dee47295e3caf8ef1f3e490476eda2866331ac541daddf WatchSource:0}: Error finding container 073ad7b07bc0b49016dee47295e3caf8ef1f3e490476eda2866331ac541daddf: Status 404 returned error can't find the container with id 073ad7b07bc0b49016dee47295e3caf8ef1f3e490476eda2866331ac541daddf
Jul 16 20:01:28.616505 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:28.616237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-97vtr" event={"ID":"de75ba51-01b2-48d4-b7f8-c0c803fa0eba","Type":"ContainerStarted","Data":"55ef2c3d56016d8da1a4b29a607237ab20731acb596bca6947b39248a594ae88"}
Jul 16 20:01:28.622163 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:28.622093    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lvjxn" event={"ID":"cc035797-bbb8-4f53-bad4-95d700ac5ce7","Type":"ContainerStarted","Data":"073ad7b07bc0b49016dee47295e3caf8ef1f3e490476eda2866331ac541daddf"}
Jul 16 20:01:28.623935 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:28.623907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9t7nr" event={"ID":"9247a48e-28a7-4b4d-aa29-9d6d65e943d8","Type":"ContainerStarted","Data":"e07b92db08fd31612b5bd29a32e974981e9e7879dbb7662828009df8113a84fb"}
Jul 16 20:01:30.632727 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:30.632642    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lvjxn" event={"ID":"cc035797-bbb8-4f53-bad4-95d700ac5ce7","Type":"ContainerStarted","Data":"b1accd7a59d24a8cc196c0ad09dee0993a6f1f80daec9a03f0385497113debdc"}
Jul 16 20:01:31.715898 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.715864    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cd48f9b78-r96df"]
Jul 16 20:01:31.726021 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.725996    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.728880 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.728680    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 20:01:31.729696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.729644    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 20:01:31.729862 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.729697    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 20:01:31.729862 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.729645    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z6795\""
Jul 16 20:01:31.730070 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.730059    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 20:01:31.730146 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.730131    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 20:01:31.730268 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.730246    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 20:01:31.730381 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.730295    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 20:01:31.731205 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.731182    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cd48f9b78-r96df"]
Jul 16 20:01:31.735783 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.735760    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 16 20:01:31.737738 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.737719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-oauth-serving-cert\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.737831 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.737751    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z87zj\" (UniqueName: \"kubernetes.io/projected/61644b21-5e63-4f64-a8fe-3742d61a8306-kube-api-access-z87zj\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.737831 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.737787    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-console-config\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.737928 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.737905    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-serving-cert\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.738014 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.737950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-oauth-config\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.738014 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.737975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-trusted-ca-bundle\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.738113 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.738014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-service-ca\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.838688 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.838653    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-oauth-config\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.838865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.838700    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-trusted-ca-bundle\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.838865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.838741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-service-ca\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.838865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.838794    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-oauth-serving-cert\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.838865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.838828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z87zj\" (UniqueName: \"kubernetes.io/projected/61644b21-5e63-4f64-a8fe-3742d61a8306-kube-api-access-z87zj\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.839072 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.838876    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-console-config\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.839072 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.838929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-serving-cert\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.840145 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.840061    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-console-config\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.840145 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.840099    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-service-ca\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.840327 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.840186    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-oauth-serving-cert\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.840327 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.840218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-trusted-ca-bundle\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.842903 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.842877    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-oauth-config\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.843124 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.843095    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-serving-cert\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:31.847138 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:31.847119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z87zj\" (UniqueName: \"kubernetes.io/projected/61644b21-5e63-4f64-a8fe-3742d61a8306-kube-api-access-z87zj\") pod \"console-5cd48f9b78-r96df\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") " pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:32.039492 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:32.039395    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:33.834756 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:33.834338    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cd48f9b78-r96df"]
Jul 16 20:01:33.838048 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:33.838000    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61644b21_5e63_4f64_a8fe_3742d61a8306.slice/crio-a8ca3c52cfdb0255198480d39cc821d86e875a9727ed3c8a0bf7edcfb88b7f43 WatchSource:0}: Error finding container a8ca3c52cfdb0255198480d39cc821d86e875a9727ed3c8a0bf7edcfb88b7f43: Status 404 returned error can't find the container with id a8ca3c52cfdb0255198480d39cc821d86e875a9727ed3c8a0bf7edcfb88b7f43
Jul 16 20:01:34.645475 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:34.645419    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-97vtr" event={"ID":"de75ba51-01b2-48d4-b7f8-c0c803fa0eba","Type":"ContainerStarted","Data":"12a4ba331f208567dcc83d3cbec7c6250a65b624009363d48fe22e36b03ea572"}
Jul 16 20:01:34.647396 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:34.647371    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lvjxn" event={"ID":"cc035797-bbb8-4f53-bad4-95d700ac5ce7","Type":"ContainerStarted","Data":"df9f846e91852c346f7afa93e2956e3a95a552722ac28ef3d1d14af50cc403af"}
Jul 16 20:01:34.647548 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:34.647473    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:34.649207 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:34.649183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9t7nr" event={"ID":"9247a48e-28a7-4b4d-aa29-9d6d65e943d8","Type":"ContainerStarted","Data":"5e28e8b929f4f30b005ee241a6791e907857956a49f44af047667afd99ddbb5c"}
Jul 16 20:01:34.650488 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:34.650468    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd48f9b78-r96df" event={"ID":"61644b21-5e63-4f64-a8fe-3742d61a8306","Type":"ContainerStarted","Data":"a8ca3c52cfdb0255198480d39cc821d86e875a9727ed3c8a0bf7edcfb88b7f43"}
Jul 16 20:01:34.664942 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:34.664894    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-97vtr" podStartSLOduration=1.801663588 podStartE2EDuration="7.664876022s" totalImagesPullingTime="5.863212434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:34.663238901 +0000 UTC m=+58.811334323" watchObservedRunningTime="2026-07-16 20:01:34.664876022 +0000 UTC m=+58.812971447"
Jul 16 20:01:34.682242 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:34.682186    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lvjxn" podStartSLOduration=5.34919222 podStartE2EDuration="7.6821699s" totalImagesPullingTime="2.33297768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:34.68049641 +0000 UTC m=+58.828591833" watchObservedRunningTime="2026-07-16 20:01:34.6821699 +0000 UTC m=+58.830265329"
Jul 16 20:01:36.558336 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:36.558307    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wtvk2"
Jul 16 20:01:37.662096 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:37.661010    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd48f9b78-r96df" event={"ID":"61644b21-5e63-4f64-a8fe-3742d61a8306","Type":"ContainerStarted","Data":"cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3"}
Jul 16 20:01:37.680764 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:37.680709    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cd48f9b78-r96df" podStartSLOduration=3.694624878 podStartE2EDuration="6.680688196s" totalImagesPullingTime="2.986063318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:37.679583851 +0000 UTC m=+61.827679277" watchObservedRunningTime="2026-07-16 20:01:37.680688196 +0000 UTC m=+61.828783619"
Jul 16 20:01:40.672519 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:40.672481    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9t7nr" event={"ID":"9247a48e-28a7-4b4d-aa29-9d6d65e943d8","Type":"ContainerStarted","Data":"2801ed2704b4aa5735bf0a0a171db515560fb767e5dbf6b46910deb6365f4d07"}
Jul 16 20:01:41.656979 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:41.656948    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lvjxn"
Jul 16 20:01:42.019591 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.019551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:42.022056 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.022035    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 20:01:42.032342 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.032320    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d822ce7-85c4-4d38-a768-6a8fcef6109f-metrics-certs\") pod \"network-metrics-daemon-spsm5\" (UID: \"3d822ce7-85c4-4d38-a768-6a8fcef6109f\") " pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:42.040250 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.040224    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:42.040362 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.040283    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:42.046586 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.046538    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:42.120836 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.120458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:42.122559 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.122535    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 20:01:42.133511 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.133487    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sfzlm\""
Jul 16 20:01:42.134020 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.133993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/381ae42e-c8c0-4763-b7c1-ce7fef4d6f64-original-pull-secret\") pod \"global-pull-secret-syncer-j665j\" (UID: \"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64\") " pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:42.141544 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.141520    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-spsm5"
Jul 16 20:01:42.221445 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.221396    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95ck\" (UniqueName: \"kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck\") pod \"network-check-target-4vxqj\" (UID: \"8a72a865-5337-4bdd-8f15-68ef40f3b6f4\") " pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:42.224077 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.224050    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 20:01:42.234623 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.234596    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 20:01:42.245378 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.245338    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q95ck\" (UniqueName: \"kubernetes.io/projected/8a72a865-5337-4bdd-8f15-68ef40f3b6f4-kube-api-access-q95ck\") pod \"network-check-target-4vxqj\" (UID: \"8a72a865-5337-4bdd-8f15-68ef40f3b6f4\") " pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:42.425473 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.425443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j665j"
Jul 16 20:01:42.439798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.439771    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nxdf6\""
Jul 16 20:01:42.447909 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.447884    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:42.684252 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:42.684169    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:01:43.720005 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:43.719970    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j665j"]
Jul 16 20:01:43.723666 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:43.723622    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod381ae42e_c8c0_4763_b7c1_ce7fef4d6f64.slice/crio-46f2937b31bab6ead55be1ebe7282c57cde83a5b40927a0cd7eb5946e15c8f1b WatchSource:0}: Error finding container 46f2937b31bab6ead55be1ebe7282c57cde83a5b40927a0cd7eb5946e15c8f1b: Status 404 returned error can't find the container with id 46f2937b31bab6ead55be1ebe7282c57cde83a5b40927a0cd7eb5946e15c8f1b
Jul 16 20:01:43.951010 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:43.950979    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4vxqj"]
Jul 16 20:01:43.953809 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:43.953784    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-spsm5"]
Jul 16 20:01:43.954332 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:43.954305    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a72a865_5337_4bdd_8f15_68ef40f3b6f4.slice/crio-ac0741adf98398792bc39115e4eacf3ee21145d807bf127dbade55a95673696e WatchSource:0}: Error finding container ac0741adf98398792bc39115e4eacf3ee21145d807bf127dbade55a95673696e: Status 404 returned error can't find the container with id ac0741adf98398792bc39115e4eacf3ee21145d807bf127dbade55a95673696e
Jul 16 20:01:43.957166 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:43.957141    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d822ce7_85c4_4d38_a768_6a8fcef6109f.slice/crio-9147093e5164af704b8d32950dc83ecacc545d08d3f19c2d08ecbac5f56b370e WatchSource:0}: Error finding container 9147093e5164af704b8d32950dc83ecacc545d08d3f19c2d08ecbac5f56b370e: Status 404 returned error can't find the container with id 9147093e5164af704b8d32950dc83ecacc545d08d3f19c2d08ecbac5f56b370e
Jul 16 20:01:44.691341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:44.691262    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4vxqj" event={"ID":"8a72a865-5337-4bdd-8f15-68ef40f3b6f4","Type":"ContainerStarted","Data":"ac0741adf98398792bc39115e4eacf3ee21145d807bf127dbade55a95673696e"}
Jul 16 20:01:44.692659 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:44.692621    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j665j" event={"ID":"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64","Type":"ContainerStarted","Data":"46f2937b31bab6ead55be1ebe7282c57cde83a5b40927a0cd7eb5946e15c8f1b"}
Jul 16 20:01:44.693974 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:44.693945    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-spsm5" event={"ID":"3d822ce7-85c4-4d38-a768-6a8fcef6109f","Type":"ContainerStarted","Data":"9147093e5164af704b8d32950dc83ecacc545d08d3f19c2d08ecbac5f56b370e"}
Jul 16 20:01:44.696497 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:44.696470    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9t7nr" event={"ID":"9247a48e-28a7-4b4d-aa29-9d6d65e943d8","Type":"ContainerStarted","Data":"6091a3c2d0b48e89f1e1902780b67ea00b6cefd84669180ec95f1f8e4139ebce"}
Jul 16 20:01:44.718270 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:44.718213    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9t7nr" podStartSLOduration=2.0986440650000002 podStartE2EDuration="17.718197119s" totalImagesPullingTime="15.619553054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:01:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:44.718092775 +0000 UTC m=+68.866188199" watchObservedRunningTime="2026-07-16 20:01:44.718197119 +0000 UTC m=+68.866292542"
Jul 16 20:01:45.704080 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:45.704041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-spsm5" event={"ID":"3d822ce7-85c4-4d38-a768-6a8fcef6109f","Type":"ContainerStarted","Data":"dcf833bc5996543360ef6835d104a81dcc64b6d2b022369157d129ad7be83f99"}
Jul 16 20:01:45.704526 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:45.704100    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-spsm5" event={"ID":"3d822ce7-85c4-4d38-a768-6a8fcef6109f","Type":"ContainerStarted","Data":"2e66782399d458632e2fcb8a58b88ae4b941f41811f16a9bbba6cc8f8138aaa9"}
Jul 16 20:01:45.724665 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:45.724587    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-spsm5" podStartSLOduration=68.801181247 podStartE2EDuration="1m9.724567251s" totalImagesPullingTime="923.386004ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:45.724384175 +0000 UTC m=+69.872479601" watchObservedRunningTime="2026-07-16 20:01:45.724567251 +0000 UTC m=+69.872662675"
Jul 16 20:01:48.108880 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.108839    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"]
Jul 16 20:01:48.112933 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.112899    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.119583 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.119557    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 16 20:01:48.119583 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.119560    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 20:01:48.119810 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.119709    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 16 20:01:48.120262 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.120230    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 20:01:48.120415 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.120305    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 20:01:48.120415 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.120351    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xb9tz\""
Jul 16 20:01:48.139221 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.139188    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"]
Jul 16 20:01:48.150424 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.150397    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"]
Jul 16 20:01:48.153064 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.153040    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.155913 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.155885    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 16 20:01:48.156579 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.156556    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nt564\""
Jul 16 20:01:48.156754 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.156730    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 16 20:01:48.156888 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.156776    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 16 20:01:48.169362 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.169335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3471f9a-d501-4ead-a27b-372dbdfd1e18-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.169518 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.169379    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3471f9a-d501-4ead-a27b-372dbdfd1e18-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.169518 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.169401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3471f9a-d501-4ead-a27b-372dbdfd1e18-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.169518 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.169509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8k5v\" (UniqueName: \"kubernetes.io/projected/c3471f9a-d501-4ead-a27b-372dbdfd1e18-kube-api-access-b8k5v\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.173071 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.173050    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lw4q4"]
Jul 16 20:01:48.175502 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.175484    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"]
Jul 16 20:01:48.175623 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.175590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.177762 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.177743    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 20:01:48.177882 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.177862    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 20:01:48.177969 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.177790    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-66zdp\""
Jul 16 20:01:48.178188 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.178174    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 20:01:48.270628 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.270596    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-wtmp\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.270628 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.270636    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqggv\" (UniqueName: \"kubernetes.io/projected/d459df04-b13a-455d-8c30-e190c2478496-kube-api-access-cqggv\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.270895 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.270663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-textfile\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.270895 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.270717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3471f9a-d501-4ead-a27b-372dbdfd1e18-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.270895 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.270757    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-tls\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.270895 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.270785    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-accelerators-collector-config\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.270895 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.270810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.270895 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:48.270826    2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 16 20:01:48.270895 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.270839    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.270895 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:48.270901    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3471f9a-d501-4ead-a27b-372dbdfd1e18-openshift-state-metrics-tls podName:c3471f9a-d501-4ead-a27b-372dbdfd1e18 nodeName:}" failed. No retries permitted until 2026-07-16 20:01:48.770880323 +0000 UTC m=+72.918975743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c3471f9a-d501-4ead-a27b-372dbdfd1e18-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-p2khk" (UID: "c3471f9a-d501-4ead-a27b-372dbdfd1e18") : secret "openshift-state-metrics-tls" not found
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.270924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d459df04-b13a-455d-8c30-e190c2478496-sys\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.270961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3471f9a-d501-4ead-a27b-372dbdfd1e18-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.271005    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.271032    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32996e00-92c4-447f-936e-b3ddbbfd50e1-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.271077    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.271120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/32996e00-92c4-447f-936e-b3ddbbfd50e1-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.271140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4sqq\" (UniqueName: \"kubernetes.io/projected/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-api-access-g4sqq\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.271165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d459df04-b13a-455d-8c30-e190c2478496-root\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.271210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3471f9a-d501-4ead-a27b-372dbdfd1e18-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.271253    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8k5v\" (UniqueName: \"kubernetes.io/projected/c3471f9a-d501-4ead-a27b-372dbdfd1e18-kube-api-access-b8k5v\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.271287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.271288    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d459df04-b13a-455d-8c30-e190c2478496-metrics-client-ca\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.272063 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.272030    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3471f9a-d501-4ead-a27b-372dbdfd1e18-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.273765 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.273742    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3471f9a-d501-4ead-a27b-372dbdfd1e18-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.280073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.280048    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8k5v\" (UniqueName: \"kubernetes.io/projected/c3471f9a-d501-4ead-a27b-372dbdfd1e18-kube-api-access-b8k5v\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.372416 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-tls\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.372608 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-accelerators-collector-config\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.372608 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372494    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.372608 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372521    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.372608 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d459df04-b13a-455d-8c30-e190c2478496-sys\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.372608 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372586    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.372886 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32996e00-92c4-447f-936e-b3ddbbfd50e1-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.372886 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.372886 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/32996e00-92c4-447f-936e-b3ddbbfd50e1-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.372886 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372807    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4sqq\" (UniqueName: \"kubernetes.io/projected/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-api-access-g4sqq\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.372886 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d459df04-b13a-455d-8c30-e190c2478496-root\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.372886 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372876    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d459df04-b13a-455d-8c30-e190c2478496-metrics-client-ca\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.373345 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:48.372887    2579 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 16 20:01:48.373345 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-wtmp\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.373345 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.372939    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqggv\" (UniqueName: \"kubernetes.io/projected/d459df04-b13a-455d-8c30-e190c2478496-kube-api-access-cqggv\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.373345 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:01:48.372975    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-tls podName:32996e00-92c4-447f-936e-b3ddbbfd50e1 nodeName:}" failed. No retries permitted until 2026-07-16 20:01:48.872952336 +0000 UTC m=+73.021047738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-2q6ts" (UID: "32996e00-92c4-447f-936e-b3ddbbfd50e1") : secret "kube-state-metrics-tls" not found
Jul 16 20:01:48.373345 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.373004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d459df04-b13a-455d-8c30-e190c2478496-sys\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.373345 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.373019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-textfile\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.373345 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.373129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-accelerators-collector-config\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.373345 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.373218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d459df04-b13a-455d-8c30-e190c2478496-root\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.373345 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.373315    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.373834 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.373388    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-textfile\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.373834 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.373401    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-wtmp\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.373834 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.373769    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d459df04-b13a-455d-8c30-e190c2478496-metrics-client-ca\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.373834 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.373774    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/32996e00-92c4-447f-936e-b3ddbbfd50e1-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.374109 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.374089    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32996e00-92c4-447f-936e-b3ddbbfd50e1-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.375288 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.375244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.375444 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.375405    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d459df04-b13a-455d-8c30-e190c2478496-node-exporter-tls\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.375996 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.375975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.382215 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.382186    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4sqq\" (UniqueName: \"kubernetes.io/projected/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-api-access-g4sqq\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.382682 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.382659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqggv\" (UniqueName: \"kubernetes.io/projected/d459df04-b13a-455d-8c30-e190c2478496-kube-api-access-cqggv\") pod \"node-exporter-lw4q4\" (UID: \"d459df04-b13a-455d-8c30-e190c2478496\") " pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.487766 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.487731    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lw4q4"
Jul 16 20:01:48.776147 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.776106    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3471f9a-d501-4ead-a27b-372dbdfd1e18-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.778512 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.778489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3471f9a-d501-4ead-a27b-372dbdfd1e18-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-p2khk\" (UID: \"c3471f9a-d501-4ead-a27b-372dbdfd1e18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:48.782692 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:48.782673    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd459df04_b13a_455d_8c30_e190c2478496.slice/crio-7952c7a619f2670943c08d026c84c266080bfa36d51f35c40d54301755233dc8 WatchSource:0}: Error finding container 7952c7a619f2670943c08d026c84c266080bfa36d51f35c40d54301755233dc8: Status 404 returned error can't find the container with id 7952c7a619f2670943c08d026c84c266080bfa36d51f35c40d54301755233dc8
Jul 16 20:01:48.876643 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.876606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:48.878862 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:48.878834    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/32996e00-92c4-447f-936e-b3ddbbfd50e1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-2q6ts\" (UID: \"32996e00-92c4-447f-936e-b3ddbbfd50e1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:49.024412 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.024327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"
Jul 16 20:01:49.063647 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.063620    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"
Jul 16 20:01:49.184649 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.184619    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk"]
Jul 16 20:01:49.188275 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:49.188239    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3471f9a_d501_4ead_a27b_372dbdfd1e18.slice/crio-ed30be43eb3e8f6250be9669de3907bc008027851f8211478729661bd9fe44ee WatchSource:0}: Error finding container ed30be43eb3e8f6250be9669de3907bc008027851f8211478729661bd9fe44ee: Status 404 returned error can't find the container with id ed30be43eb3e8f6250be9669de3907bc008027851f8211478729661bd9fe44ee
Jul 16 20:01:49.204006 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.203959    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 20:01:49.228058 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.228024    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 20:01:49.228283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.228186    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.230950 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.230602    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 16 20:01:49.230950 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.230645    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 16 20:01:49.230950 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.230651    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 16 20:01:49.230950 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.230771    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 16 20:01:49.230950 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.230783    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 16 20:01:49.231916 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.231528    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 16 20:01:49.231916 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.231603    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 16 20:01:49.231916 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.231604    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 16 20:01:49.231916 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.231636    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 16 20:01:49.231916 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.231608    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8gnxc\""
Jul 16 20:01:49.244014 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.243986    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-2q6ts"]
Jul 16 20:01:49.247501 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:49.247474    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32996e00_92c4_447f_936e_b3ddbbfd50e1.slice/crio-f98001050302e00f959e857ac7758cd7eeae81151cba4dd202c3f222d88bc2b8 WatchSource:0}: Error finding container f98001050302e00f959e857ac7758cd7eeae81151cba4dd202c3f222d88bc2b8: Status 404 returned error can't find the container with id f98001050302e00f959e857ac7758cd7eeae81151cba4dd202c3f222d88bc2b8
Jul 16 20:01:49.280450 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280450 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280416    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-volume\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280622 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280659 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280647    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280692 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280725 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280690    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-web-config\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280725 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280788 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280788 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280762    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280788 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280781    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280872 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280798    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hbzd\" (UniqueName: \"kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-kube-api-access-2hbzd\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280872 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-out\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.280931 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.280868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.381960 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.381786    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-out\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.381960 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.381829    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.381960 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.381883    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.381960 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.381926    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-volume\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.381960 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.381955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.382352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.381983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.382352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.382085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.382352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.382124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-web-config\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.382352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.382153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.382352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.382176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.382352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.382211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.382352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.382240    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.382352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.382268    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hbzd\" (UniqueName: \"kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-kube-api-access-2hbzd\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.384078 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.383460    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.384078 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.383627    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.384668 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.384259    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.385601 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.385573    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-out\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.387786 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.387757    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.388709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.388510    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.388709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.388526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.388709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.388628    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-volume\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.388709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.388644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.388709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.388667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.389732 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.389706    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-web-config\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.390226 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.390204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.391657 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.391634    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hbzd\" (UniqueName: \"kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-kube-api-access-2hbzd\") pod \"alertmanager-main-0\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.552110 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.552022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:01:49.708933 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.708896    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 20:01:49.719763 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.719661    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4vxqj" event={"ID":"8a72a865-5337-4bdd-8f15-68ef40f3b6f4","Type":"ContainerStarted","Data":"02960ea216f06d277f6e47353f13168267c91eea447624c5b12747f0fba42ecd"}
Jul 16 20:01:49.719763 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.719730    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:01:49.721282 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.721253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j665j" event={"ID":"381ae42e-c8c0-4763-b7c1-ce7fef4d6f64","Type":"ContainerStarted","Data":"4046163d24c046e095d9e4a2d018c96bcd2bd543e383828379fd2877d67d119a"}
Jul 16 20:01:49.723385 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.723354    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk" event={"ID":"c3471f9a-d501-4ead-a27b-372dbdfd1e18","Type":"ContainerStarted","Data":"5d31e1d083ec99ce68d0ddea4df91d41e82c7f042dc4c34ffbf520a092b3d05e"}
Jul 16 20:01:49.723522 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.723420    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk" event={"ID":"c3471f9a-d501-4ead-a27b-372dbdfd1e18","Type":"ContainerStarted","Data":"1af0a8d5c895a273578e34dabc51dad03df20607989c80f01c577cca2b3ca5df"}
Jul 16 20:01:49.723522 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.723470    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk" event={"ID":"c3471f9a-d501-4ead-a27b-372dbdfd1e18","Type":"ContainerStarted","Data":"ed30be43eb3e8f6250be9669de3907bc008027851f8211478729661bd9fe44ee"}
Jul 16 20:01:49.724659 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.724636    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts" event={"ID":"32996e00-92c4-447f-936e-b3ddbbfd50e1","Type":"ContainerStarted","Data":"f98001050302e00f959e857ac7758cd7eeae81151cba4dd202c3f222d88bc2b8"}
Jul 16 20:01:49.725823 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.725799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lw4q4" event={"ID":"d459df04-b13a-455d-8c30-e190c2478496","Type":"ContainerStarted","Data":"7952c7a619f2670943c08d026c84c266080bfa36d51f35c40d54301755233dc8"}
Jul 16 20:01:49.736057 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.735833    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4vxqj" podStartSLOduration=68.901625449 podStartE2EDuration="1m13.735812537s" totalImagesPullingTime="4.834187088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:49.735466482 +0000 UTC m=+73.883561906" watchObservedRunningTime="2026-07-16 20:01:49.735812537 +0000 UTC m=+73.883907962"
Jul 16 20:01:49.751390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:49.751331    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j665j" podStartSLOduration=68.695926988 podStartE2EDuration="1m13.75131333s" totalImagesPullingTime="5.055386342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:49.750349263 +0000 UTC m=+73.898444686" watchObservedRunningTime="2026-07-16 20:01:49.75131333 +0000 UTC m=+73.899408753"
Jul 16 20:01:49.910248 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:49.910175    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d098bd_ff6b_43f6_8299_60b27a454c8d.slice/crio-27c18b35e175883a1b90283a7e3d676963512dbe1c02af29a0cd986adc18f803 WatchSource:0}: Error finding container 27c18b35e175883a1b90283a7e3d676963512dbe1c02af29a0cd986adc18f803: Status 404 returned error can't find the container with id 27c18b35e175883a1b90283a7e3d676963512dbe1c02af29a0cd986adc18f803
Jul 16 20:01:50.170523 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.170492    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-764c4bdfd-zhln4"]
Jul 16 20:01:50.173307 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.173284    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.175572 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.175550    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 16 20:01:50.175719 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.175690    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fmlsgu2mn78dc\""
Jul 16 20:01:50.175828 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.175730    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 16 20:01:50.175828 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.175760    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 16 20:01:50.175828 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.175760    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 16 20:01:50.175828 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.175762    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fpr6m\""
Jul 16 20:01:50.175828 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.175690    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 16 20:01:50.185329 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.185304    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-764c4bdfd-zhln4"]
Jul 16 20:01:50.291105 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.291024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-tls\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.291283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.291113    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.291283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.291140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa9b886a-6291-4502-ac97-d140b1853806-metrics-client-ca\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.291283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.291169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv89d\" (UniqueName: \"kubernetes.io/projected/aa9b886a-6291-4502-ac97-d140b1853806-kube-api-access-jv89d\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.291283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.291198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.291283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.291228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.291283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.291264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.291672 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.291313    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-grpc-tls\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.392877 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.392640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.392877 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.392697    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.392877 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.392749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-grpc-tls\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.392877 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.392798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-tls\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.392877 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.392858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.392877 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.392881    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa9b886a-6291-4502-ac97-d140b1853806-metrics-client-ca\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.393292 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.392909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv89d\" (UniqueName: \"kubernetes.io/projected/aa9b886a-6291-4502-ac97-d140b1853806-kube-api-access-jv89d\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.393292 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.392955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.394338 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.394103    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa9b886a-6291-4502-ac97-d140b1853806-metrics-client-ca\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.395693 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.395644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.397122 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.397095    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-tls\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.397324 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.397277    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.398022 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.397982    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-grpc-tls\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.401454 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.398830    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.401454 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.399259    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa9b886a-6291-4502-ac97-d140b1853806-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.406162 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.406119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv89d\" (UniqueName: \"kubernetes.io/projected/aa9b886a-6291-4502-ac97-d140b1853806-kube-api-access-jv89d\") pod \"thanos-querier-764c4bdfd-zhln4\" (UID: \"aa9b886a-6291-4502-ac97-d140b1853806\") " pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.494416 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.494316    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:50.734205 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.734171    2579 generic.go:356] "Generic (PLEG): container finished" podID="d459df04-b13a-455d-8c30-e190c2478496" containerID="da00ca87e695ba93076c2a00a680803754d83ebc51a40f4ed2750bedec479231" exitCode=0
Jul 16 20:01:50.734318 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.734268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lw4q4" event={"ID":"d459df04-b13a-455d-8c30-e190c2478496","Type":"ContainerDied","Data":"da00ca87e695ba93076c2a00a680803754d83ebc51a40f4ed2750bedec479231"}
Jul 16 20:01:50.736484 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.736368    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerStarted","Data":"27c18b35e175883a1b90283a7e3d676963512dbe1c02af29a0cd986adc18f803"}
Jul 16 20:01:50.791215 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:50.791082    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-764c4bdfd-zhln4"]
Jul 16 20:01:50.995001 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:50.994960    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9b886a_6291_4502_ac97_d140b1853806.slice/crio-505a44ffe2e0122af7d000c92c7b1c534aff9fbc3a643bbee895135fe4f0b03f WatchSource:0}: Error finding container 505a44ffe2e0122af7d000c92c7b1c534aff9fbc3a643bbee895135fe4f0b03f: Status 404 returned error can't find the container with id 505a44ffe2e0122af7d000c92c7b1c534aff9fbc3a643bbee895135fe4f0b03f
Jul 16 20:01:51.744212 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:51.744170    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk" event={"ID":"c3471f9a-d501-4ead-a27b-372dbdfd1e18","Type":"ContainerStarted","Data":"6a829c8e7c0cdc69e8642c9e4a9a8627fe97d841dfbb4cc115b8572f1b067e95"}
Jul 16 20:01:51.747758 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:51.747726    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts" event={"ID":"32996e00-92c4-447f-936e-b3ddbbfd50e1","Type":"ContainerStarted","Data":"b0c2a303be387f8dcfa8a6f04ce60e1ee609039dc89c37e25b0353798c37188b"}
Jul 16 20:01:51.747878 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:51.747801    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts" event={"ID":"32996e00-92c4-447f-936e-b3ddbbfd50e1","Type":"ContainerStarted","Data":"70278608a475f2c859d599681daf6e2e02ce7c6e821e04c0816be6b87ddcba67"}
Jul 16 20:01:51.747878 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:51.747829    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts" event={"ID":"32996e00-92c4-447f-936e-b3ddbbfd50e1","Type":"ContainerStarted","Data":"a5abe14606d36f38be5415375673bbb9ecb5ff1f9836bc2ed8d8372fa129f85a"}
Jul 16 20:01:51.750836 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:51.750753    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lw4q4" event={"ID":"d459df04-b13a-455d-8c30-e190c2478496","Type":"ContainerStarted","Data":"6dd30062a567607f16159ed48f1b5b46ecdcde040d165361afc334d722d1a590"}
Jul 16 20:01:51.750836 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:51.750804    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lw4q4" event={"ID":"d459df04-b13a-455d-8c30-e190c2478496","Type":"ContainerStarted","Data":"5869f005ca7e8695f1024bf6669f59dec45cefd5d56d4bc80fb88d2a389907ef"}
Jul 16 20:01:51.752122 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:51.752068    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4" event={"ID":"aa9b886a-6291-4502-ac97-d140b1853806","Type":"ContainerStarted","Data":"505a44ffe2e0122af7d000c92c7b1c534aff9fbc3a643bbee895135fe4f0b03f"}
Jul 16 20:01:51.762108 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:51.762056    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-p2khk" podStartSLOduration=2.246598995 podStartE2EDuration="3.762039487s" totalImagesPullingTime="1.515440492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:51.761593303 +0000 UTC m=+75.909688727" watchObservedRunningTime="2026-07-16 20:01:51.762039487 +0000 UTC m=+75.910134910"
Jul 16 20:01:51.780161 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:51.779939    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lw4q4" podStartSLOduration=2.323308762 podStartE2EDuration="3.779920891s" totalImagesPullingTime="1.374356007s" totalInitContainerRuntime="82.256122ms" isStatefulPod=true podCreationTimestamp="2026-07-16 20:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:51.778188103 +0000 UTC m=+75.926283527" watchObservedRunningTime="2026-07-16 20:01:51.779920891 +0000 UTC m=+75.928016317"
Jul 16 20:01:51.798843 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:51.798781    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-2q6ts" podStartSLOduration=2.024633903 podStartE2EDuration="3.798769567s" totalImagesPullingTime="1.774135664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:51.798135917 +0000 UTC m=+75.946231341" watchObservedRunningTime="2026-07-16 20:01:51.798769567 +0000 UTC m=+75.946864991"
Jul 16 20:01:52.662408 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.662367    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-749644bcf7-f6frz"]
Jul 16 20:01:52.665360 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.665336    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.667601 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.667576    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 20:01:52.667718 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.667610    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 16 20:01:52.667783 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.667719    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jgwh8\""
Jul 16 20:01:52.668604 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.668581    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 16 20:01:52.668723 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.668625    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 16 20:01:52.668723 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.668583    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8lrjlopo8g8q\""
Jul 16 20:01:52.676107 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.676084    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-749644bcf7-f6frz"]
Jul 16 20:01:52.757934 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.757900    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerID="3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02" exitCode=0
Jul 16 20:01:52.758374 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.757990    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerDied","Data":"3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02"}
Jul 16 20:01:52.813239 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.813211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm6hk\" (UniqueName: \"kubernetes.io/projected/e0972057-5af7-41b5-9ac4-ca57a836167a-kube-api-access-dm6hk\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.813427 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.813301    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0972057-5af7-41b5-9ac4-ca57a836167a-client-ca-bundle\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.813427 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.813343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0972057-5af7-41b5-9ac4-ca57a836167a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.813427 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.813380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e0972057-5af7-41b5-9ac4-ca57a836167a-metrics-server-audit-profiles\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.813427 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.813412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e0972057-5af7-41b5-9ac4-ca57a836167a-secret-metrics-server-client-certs\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.813635 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.813463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e0972057-5af7-41b5-9ac4-ca57a836167a-secret-metrics-server-tls\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.813635 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.813485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e0972057-5af7-41b5-9ac4-ca57a836167a-audit-log\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.880259 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.880224    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"]
Jul 16 20:01:52.882688 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.882670    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"
Jul 16 20:01:52.884931 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.884904    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 16 20:01:52.885040 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.884980    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-nxvcv\""
Jul 16 20:01:52.887575 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.887553    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-974f7fcf5-62dcq"]
Jul 16 20:01:52.889749 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.889722    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:52.894862 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.894840    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"]
Jul 16 20:01:52.901383 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.901359    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-974f7fcf5-62dcq"]
Jul 16 20:01:52.914740 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.914680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0972057-5af7-41b5-9ac4-ca57a836167a-client-ca-bundle\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.914878 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.914805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0972057-5af7-41b5-9ac4-ca57a836167a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.914939 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.914901    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e0972057-5af7-41b5-9ac4-ca57a836167a-metrics-server-audit-profiles\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.914992 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.914959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e0972057-5af7-41b5-9ac4-ca57a836167a-secret-metrics-server-client-certs\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.915285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.915248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e0972057-5af7-41b5-9ac4-ca57a836167a-secret-metrics-server-tls\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.915390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.915288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e0972057-5af7-41b5-9ac4-ca57a836167a-audit-log\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.915390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.915357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm6hk\" (UniqueName: \"kubernetes.io/projected/e0972057-5af7-41b5-9ac4-ca57a836167a-kube-api-access-dm6hk\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.916348 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.916322    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0972057-5af7-41b5-9ac4-ca57a836167a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.916482 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.916397    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e0972057-5af7-41b5-9ac4-ca57a836167a-audit-log\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.916713 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.916672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e0972057-5af7-41b5-9ac4-ca57a836167a-metrics-server-audit-profiles\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.918087 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.918063    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0972057-5af7-41b5-9ac4-ca57a836167a-client-ca-bundle\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.918522 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.918501    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e0972057-5af7-41b5-9ac4-ca57a836167a-secret-metrics-server-client-certs\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.918802 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.918785    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e0972057-5af7-41b5-9ac4-ca57a836167a-secret-metrics-server-tls\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.924743 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.924710    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm6hk\" (UniqueName: \"kubernetes.io/projected/e0972057-5af7-41b5-9ac4-ca57a836167a-kube-api-access-dm6hk\") pod \"metrics-server-749644bcf7-f6frz\" (UID: \"e0972057-5af7-41b5-9ac4-ca57a836167a\") " pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:52.976603 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:52.976553    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:01:53.016425 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.016387    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-trusted-ca-bundle\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.016588 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.016484    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-service-ca\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.016645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.016585    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e0157262-e344-4390-a1cf-df0028852271-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-8vkv4\" (UID: \"e0157262-e344-4390-a1cf-df0028852271\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"
Jul 16 20:01:53.016645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.016618    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-oauth-config\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.016645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.016640    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-console-config\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.016756 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.016693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-oauth-serving-cert\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.016756 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.016721    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r9z2\" (UniqueName: \"kubernetes.io/projected/dc4f8142-6d5e-4ba6-93db-171d28427222-kube-api-access-6r9z2\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.016855 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.016760    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v44kk\" (UniqueName: \"kubernetes.io/projected/e0157262-e344-4390-a1cf-df0028852271-kube-api-access-v44kk\") pod \"monitoring-plugin-7b7b7bc8bf-8vkv4\" (UID: \"e0157262-e344-4390-a1cf-df0028852271\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"
Jul 16 20:01:53.016855 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.016785    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-serving-cert\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.117531 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.117504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e0157262-e344-4390-a1cf-df0028852271-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-8vkv4\" (UID: \"e0157262-e344-4390-a1cf-df0028852271\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"
Jul 16 20:01:53.117641 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.117544    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-oauth-config\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.117641 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.117574    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-console-config\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.118027 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.117998    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-oauth-serving-cert\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.118117 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.118056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r9z2\" (UniqueName: \"kubernetes.io/projected/dc4f8142-6d5e-4ba6-93db-171d28427222-kube-api-access-6r9z2\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.118179 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.118125    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v44kk\" (UniqueName: \"kubernetes.io/projected/e0157262-e344-4390-a1cf-df0028852271-kube-api-access-v44kk\") pod \"monitoring-plugin-7b7b7bc8bf-8vkv4\" (UID: \"e0157262-e344-4390-a1cf-df0028852271\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"
Jul 16 20:01:53.118179 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.118154    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-serving-cert\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.118268 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.118187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-trusted-ca-bundle\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.118268 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.118223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-console-config\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.118268 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.118242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-service-ca\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.118698 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.118668    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-oauth-serving-cert\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.119071 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.119035    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-service-ca\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.119942 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.119889    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-trusted-ca-bundle\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.120499 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.120463    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-oauth-config\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.120904 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.120873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e0157262-e344-4390-a1cf-df0028852271-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-8vkv4\" (UID: \"e0157262-e344-4390-a1cf-df0028852271\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"
Jul 16 20:01:53.121301 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.121283    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-serving-cert\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.126003 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.125971    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v44kk\" (UniqueName: \"kubernetes.io/projected/e0157262-e344-4390-a1cf-df0028852271-kube-api-access-v44kk\") pod \"monitoring-plugin-7b7b7bc8bf-8vkv4\" (UID: \"e0157262-e344-4390-a1cf-df0028852271\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"
Jul 16 20:01:53.126624 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.126600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r9z2\" (UniqueName: \"kubernetes.io/projected/dc4f8142-6d5e-4ba6-93db-171d28427222-kube-api-access-6r9z2\") pod \"console-974f7fcf5-62dcq\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") " pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.163724 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.163698    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-749644bcf7-f6frz"]
Jul 16 20:01:53.166941 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:53.166910    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0972057_5af7_41b5_9ac4_ca57a836167a.slice/crio-1a22cfa65b37d4efb9c1e1d09861711ff3921af8747ce1516d4d3e1c678d995d WatchSource:0}: Error finding container 1a22cfa65b37d4efb9c1e1d09861711ff3921af8747ce1516d4d3e1c678d995d: Status 404 returned error can't find the container with id 1a22cfa65b37d4efb9c1e1d09861711ff3921af8747ce1516d4d3e1c678d995d
Jul 16 20:01:53.194128 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.194111    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"
Jul 16 20:01:53.202121 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.202082    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:01:53.347500 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.347444    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"]
Jul 16 20:01:53.350345 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.350320    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.353391 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.352499    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 16 20:01:53.353391 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.352551    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 16 20:01:53.353391 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.352907    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-k69vb\""
Jul 16 20:01:53.353391 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.352969    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 16 20:01:53.353391 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.353154    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 16 20:01:53.353391 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.353225    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 16 20:01:53.366539 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.366501    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 16 20:01:53.368919 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.368889    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"]
Jul 16 20:01:53.379643 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.379621    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"]
Jul 16 20:01:53.383741 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:53.383713    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0157262_e344_4390_a1cf_df0028852271.slice/crio-7e0624aa3d0e73f2f2bd1b6953e034f72a28301066189e9ca55e30acfdc9ef88 WatchSource:0}: Error finding container 7e0624aa3d0e73f2f2bd1b6953e034f72a28301066189e9ca55e30acfdc9ef88: Status 404 returned error can't find the container with id 7e0624aa3d0e73f2f2bd1b6953e034f72a28301066189e9ca55e30acfdc9ef88
Jul 16 20:01:53.393824 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.393802    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-974f7fcf5-62dcq"]
Jul 16 20:01:53.397100 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:53.397075    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc4f8142_6d5e_4ba6_93db_171d28427222.slice/crio-0f8f2ec442c12a55ddec3e93dd167637be6901c57056739cd94a14cba54e19ba WatchSource:0}: Error finding container 0f8f2ec442c12a55ddec3e93dd167637be6901c57056739cd94a14cba54e19ba: Status 404 returned error can't find the container with id 0f8f2ec442c12a55ddec3e93dd167637be6901c57056739cd94a14cba54e19ba
Jul 16 20:01:53.421469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.421423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-federate-client-tls\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.421563 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.421506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skbd2\" (UniqueName: \"kubernetes.io/projected/fef3b439-79a8-4245-90dc-968f429abe7d-kube-api-access-skbd2\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.421636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.421583    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef3b439-79a8-4245-90dc-968f429abe7d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.421689 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.421631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fef3b439-79a8-4245-90dc-968f429abe7d-metrics-client-ca\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.421689 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.421670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-telemeter-client-tls\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.421785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.421714    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-secret-telemeter-client\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.421785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.421743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.421785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.421776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef3b439-79a8-4245-90dc-968f429abe7d-serving-certs-ca-bundle\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.522169 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.522136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.522169 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.522176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef3b439-79a8-4245-90dc-968f429abe7d-serving-certs-ca-bundle\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.522402 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.522200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-federate-client-tls\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.522402 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.522239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbd2\" (UniqueName: \"kubernetes.io/projected/fef3b439-79a8-4245-90dc-968f429abe7d-kube-api-access-skbd2\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.522402 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.522292    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef3b439-79a8-4245-90dc-968f429abe7d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.522402 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.522324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fef3b439-79a8-4245-90dc-968f429abe7d-metrics-client-ca\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.522402 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.522358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-telemeter-client-tls\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.522402 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.522401    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-secret-telemeter-client\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.523057 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.523003    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef3b439-79a8-4245-90dc-968f429abe7d-serving-certs-ca-bundle\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.523184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.523129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fef3b439-79a8-4245-90dc-968f429abe7d-metrics-client-ca\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.523357 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.523332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef3b439-79a8-4245-90dc-968f429abe7d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.525567 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.525522    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-secret-telemeter-client\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.525840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.525786    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.525840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.525791    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-federate-client-tls\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.526640 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.526622    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fef3b439-79a8-4245-90dc-968f429abe7d-telemeter-client-tls\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.530879 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.530857    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skbd2\" (UniqueName: \"kubernetes.io/projected/fef3b439-79a8-4245-90dc-968f429abe7d-kube-api-access-skbd2\") pod \"telemeter-client-56c4c79f8d-kw55p\" (UID: \"fef3b439-79a8-4245-90dc-968f429abe7d\") " pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.667527 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.667493    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"
Jul 16 20:01:53.765355 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.765316    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-974f7fcf5-62dcq" event={"ID":"dc4f8142-6d5e-4ba6-93db-171d28427222","Type":"ContainerStarted","Data":"84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16"}
Jul 16 20:01:53.765814 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.765386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-974f7fcf5-62dcq" event={"ID":"dc4f8142-6d5e-4ba6-93db-171d28427222","Type":"ContainerStarted","Data":"0f8f2ec442c12a55ddec3e93dd167637be6901c57056739cd94a14cba54e19ba"}
Jul 16 20:01:53.773168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.770752    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4" event={"ID":"e0157262-e344-4390-a1cf-df0028852271","Type":"ContainerStarted","Data":"7e0624aa3d0e73f2f2bd1b6953e034f72a28301066189e9ca55e30acfdc9ef88"}
Jul 16 20:01:53.773955 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.773862    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-749644bcf7-f6frz" event={"ID":"e0972057-5af7-41b5-9ac4-ca57a836167a","Type":"ContainerStarted","Data":"1a22cfa65b37d4efb9c1e1d09861711ff3921af8747ce1516d4d3e1c678d995d"}
Jul 16 20:01:53.778352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.778102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4" event={"ID":"aa9b886a-6291-4502-ac97-d140b1853806","Type":"ContainerStarted","Data":"e28842c867c086a34d82d4ea1b71d64648304d0a165b9a8fdd59a44851899431"}
Jul 16 20:01:53.778352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.778175    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4" event={"ID":"aa9b886a-6291-4502-ac97-d140b1853806","Type":"ContainerStarted","Data":"b185fbdfe9299fc4b833fb69f90d037a072c32bfd97c2a200ad16c9004fe14a2"}
Jul 16 20:01:53.778352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.778219    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4" event={"ID":"aa9b886a-6291-4502-ac97-d140b1853806","Type":"ContainerStarted","Data":"754912a2d179c8c9865271b153e533e0f975358f60aff6cac3bbc150347fde18"}
Jul 16 20:01:53.790530 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.789315    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-974f7fcf5-62dcq" podStartSLOduration=1.7892985540000002 podStartE2EDuration="1.789298554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:53.788721617 +0000 UTC m=+77.936817040" watchObservedRunningTime="2026-07-16 20:01:53.789298554 +0000 UTC m=+77.937393980"
Jul 16 20:01:53.832258 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:53.832162    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-56c4c79f8d-kw55p"]
Jul 16 20:01:53.834620 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:01:53.834591    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef3b439_79a8_4245_90dc_968f429abe7d.slice/crio-df846a0e7c9bdd70ee0d440db9f2ccb3a15747b7643aa48b09c50c439c5308ca WatchSource:0}: Error finding container df846a0e7c9bdd70ee0d440db9f2ccb3a15747b7643aa48b09c50c439c5308ca: Status 404 returned error can't find the container with id df846a0e7c9bdd70ee0d440db9f2ccb3a15747b7643aa48b09c50c439c5308ca
Jul 16 20:01:54.784168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:54.784133    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p" event={"ID":"fef3b439-79a8-4245-90dc-968f429abe7d","Type":"ContainerStarted","Data":"df846a0e7c9bdd70ee0d440db9f2ccb3a15747b7643aa48b09c50c439c5308ca"}
Jul 16 20:01:55.792772 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.792681    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerStarted","Data":"97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc"}
Jul 16 20:01:55.793196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.792804    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerStarted","Data":"7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e"}
Jul 16 20:01:55.793196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.792845    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerStarted","Data":"f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e"}
Jul 16 20:01:55.793196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.792882    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerStarted","Data":"ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb"}
Jul 16 20:01:55.793196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.792917    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerStarted","Data":"462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3"}
Jul 16 20:01:55.793196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.792953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerStarted","Data":"f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4"}
Jul 16 20:01:55.795630 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.795604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4" event={"ID":"aa9b886a-6291-4502-ac97-d140b1853806","Type":"ContainerStarted","Data":"7fb3d81c7db60755d801163da94498b03ad78729a232f60705f5585f117b4de5"}
Jul 16 20:01:55.795757 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.795679    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4" event={"ID":"aa9b886a-6291-4502-ac97-d140b1853806","Type":"ContainerStarted","Data":"faf43b5060c70c301082c017ce876aef3ca3ac30fe5bf68d94b8389a90019b3b"}
Jul 16 20:01:55.795757 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.795727    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4" event={"ID":"aa9b886a-6291-4502-ac97-d140b1853806","Type":"ContainerStarted","Data":"f6f0d25068a28976e42011d3262b7f358230c4e3483c2913a3ee713295de32b1"}
Jul 16 20:01:55.795851 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.795773    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:01:55.797167 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.797146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4" event={"ID":"e0157262-e344-4390-a1cf-df0028852271","Type":"ContainerStarted","Data":"4ee9efb53dc5c7af2dfb60071f98d6f4941f4a50b3c8219ed82c5b5130c4e2aa"}
Jul 16 20:01:55.797306 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.797287    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"
Jul 16 20:01:55.798828 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.798798    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-749644bcf7-f6frz" event={"ID":"e0972057-5af7-41b5-9ac4-ca57a836167a","Type":"ContainerStarted","Data":"b4d223803a69a07e991ff9db6967e474f5a03e2f8441cb8177fd7bd48a40c508"}
Jul 16 20:01:55.803480 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.803459    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4"
Jul 16 20:01:55.819744 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.819700    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.289089469 podStartE2EDuration="6.819687425s" totalImagesPullingTime="4.510807303s" totalInitContainerRuntime="19.790653ms" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:55.818939761 +0000 UTC m=+79.967035213" watchObservedRunningTime="2026-07-16 20:01:55.819687425 +0000 UTC m=+79.967782841"
Jul 16 20:01:55.841327 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.841274    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-749644bcf7-f6frz" podStartSLOduration=1.761402167 podStartE2EDuration="3.841253579s" totalImagesPullingTime="2.079851412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:55.838000203 +0000 UTC m=+79.986095642" watchObservedRunningTime="2026-07-16 20:01:55.841253579 +0000 UTC m=+79.989349004"
Jul 16 20:01:55.856677 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.856624    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8vkv4" podStartSLOduration=1.968329319 podStartE2EDuration="3.85660507s" totalImagesPullingTime="1.888275751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:55.855383527 +0000 UTC m=+80.003478969" watchObservedRunningTime="2026-07-16 20:01:55.85660507 +0000 UTC m=+80.004700497"
Jul 16 20:01:55.877039 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:55.876990    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4" podStartSLOduration=1.8332497970000001 podStartE2EDuration="5.876971198s" totalImagesPullingTime="4.043721401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:55.875915467 +0000 UTC m=+80.024010918" watchObservedRunningTime="2026-07-16 20:01:55.876971198 +0000 UTC m=+80.025066622"
Jul 16 20:01:56.804398 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:56.804303    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p" event={"ID":"fef3b439-79a8-4245-90dc-968f429abe7d","Type":"ContainerStarted","Data":"fc0ded51786663cc3b2e4762e7faa0934c06399a9f4a4f19ffaf7cb7c593fc8c"}
Jul 16 20:01:56.804398 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:56.804395    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p" event={"ID":"fef3b439-79a8-4245-90dc-968f429abe7d","Type":"ContainerStarted","Data":"9d515811cfee9f2f14e996b25f15b522a99128905a032184fdd169a9c24a6c50"}
Jul 16 20:01:56.804860 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:56.804420    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p" event={"ID":"fef3b439-79a8-4245-90dc-968f429abe7d","Type":"ContainerStarted","Data":"610fc0f214feb92533aa43e95afac6d17862d5d4a009e1a4a20c9e9fd5b1432d"}
Jul 16 20:01:56.831870 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:56.831824    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-56c4c79f8d-kw55p" podStartSLOduration=1.17567168 podStartE2EDuration="3.83181124s" totalImagesPullingTime="2.65613956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:01:56.830248349 +0000 UTC m=+80.978343774" watchObservedRunningTime="2026-07-16 20:01:56.83181124 +0000 UTC m=+80.979906662"
Jul 16 20:01:57.623486 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:01:57.623452    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-974f7fcf5-62dcq"]
Jul 16 20:02:01.812820 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:01.812672    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-764c4bdfd-zhln4"
Jul 16 20:02:03.203168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:03.203127    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:02:08.935342 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:08.935302    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cd48f9b78-r96df"]
Jul 16 20:02:12.977547 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:12.977511    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:02:12.977945 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:12.977575    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:02:20.739480 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:20.739422    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4vxqj"
Jul 16 20:02:21.318551 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:21.318518    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-97vtr_de75ba51-01b2-48d4-b7f8-c0c803fa0eba/serve-healthcheck-canary/0.log"
Jul 16 20:02:22.643423 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:22.643387    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-974f7fcf5-62dcq" podUID="dc4f8142-6d5e-4ba6-93db-171d28427222" containerName="console" containerID="cri-o://84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16" gracePeriod=15
Jul 16 20:02:23.407824 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.407804    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:02:23.483780 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.483751    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-oauth-config\") pod \"dc4f8142-6d5e-4ba6-93db-171d28427222\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") "
Jul 16 20:02:23.483927 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.483792    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-console-config\") pod \"dc4f8142-6d5e-4ba6-93db-171d28427222\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") "
Jul 16 20:02:23.483927 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.483826    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-serving-cert\") pod \"dc4f8142-6d5e-4ba6-93db-171d28427222\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") "
Jul 16 20:02:23.484046 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.483922    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r9z2\" (UniqueName: \"kubernetes.io/projected/dc4f8142-6d5e-4ba6-93db-171d28427222-kube-api-access-6r9z2\") pod \"dc4f8142-6d5e-4ba6-93db-171d28427222\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") "
Jul 16 20:02:23.484046 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.483986    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-trusted-ca-bundle\") pod \"dc4f8142-6d5e-4ba6-93db-171d28427222\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") "
Jul 16 20:02:23.484046 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.484036    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-oauth-serving-cert\") pod \"dc4f8142-6d5e-4ba6-93db-171d28427222\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") "
Jul 16 20:02:23.484192 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.484072    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-service-ca\") pod \"dc4f8142-6d5e-4ba6-93db-171d28427222\" (UID: \"dc4f8142-6d5e-4ba6-93db-171d28427222\") "
Jul 16 20:02:23.484346 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.484289    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-console-config" (OuterVolumeSpecName: "console-config") pod "dc4f8142-6d5e-4ba6-93db-171d28427222" (UID: "dc4f8142-6d5e-4ba6-93db-171d28427222"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:02:23.484513 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.484373    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-console-config\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:23.484513 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.484424    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dc4f8142-6d5e-4ba6-93db-171d28427222" (UID: "dc4f8142-6d5e-4ba6-93db-171d28427222"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:02:23.484513 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.484452    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dc4f8142-6d5e-4ba6-93db-171d28427222" (UID: "dc4f8142-6d5e-4ba6-93db-171d28427222"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:02:23.484667 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.484505    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-service-ca" (OuterVolumeSpecName: "service-ca") pod "dc4f8142-6d5e-4ba6-93db-171d28427222" (UID: "dc4f8142-6d5e-4ba6-93db-171d28427222"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:02:23.486122 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.486090    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dc4f8142-6d5e-4ba6-93db-171d28427222" (UID: "dc4f8142-6d5e-4ba6-93db-171d28427222"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:02:23.486227 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.486144    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dc4f8142-6d5e-4ba6-93db-171d28427222" (UID: "dc4f8142-6d5e-4ba6-93db-171d28427222"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:02:23.486227 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.486153    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc4f8142-6d5e-4ba6-93db-171d28427222-kube-api-access-6r9z2" (OuterVolumeSpecName: "kube-api-access-6r9z2") pod "dc4f8142-6d5e-4ba6-93db-171d28427222" (UID: "dc4f8142-6d5e-4ba6-93db-171d28427222"). InnerVolumeSpecName "kube-api-access-6r9z2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:02:23.585071 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.585041    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r9z2\" (UniqueName: \"kubernetes.io/projected/dc4f8142-6d5e-4ba6-93db-171d28427222-kube-api-access-6r9z2\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:23.585071 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.585069    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-trusted-ca-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:23.585071 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.585079    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-oauth-serving-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:23.585294 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.585088    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc4f8142-6d5e-4ba6-93db-171d28427222-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:23.585294 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.585097    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-oauth-config\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:23.585294 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.585106    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc4f8142-6d5e-4ba6-93db-171d28427222-console-serving-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:23.919306 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.919265    2579 generic.go:356] "Generic (PLEG): container finished" podID="dc4f8142-6d5e-4ba6-93db-171d28427222" containerID="84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16" exitCode=0
Jul 16 20:02:23.919742 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.919315    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-974f7fcf5-62dcq"
Jul 16 20:02:23.919742 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.919347    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-974f7fcf5-62dcq" event={"ID":"dc4f8142-6d5e-4ba6-93db-171d28427222","Type":"ContainerDied","Data":"84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16"}
Jul 16 20:02:23.919742 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.919403    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-974f7fcf5-62dcq" event={"ID":"dc4f8142-6d5e-4ba6-93db-171d28427222","Type":"ContainerDied","Data":"0f8f2ec442c12a55ddec3e93dd167637be6901c57056739cd94a14cba54e19ba"}
Jul 16 20:02:23.919742 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.919425    2579 scope.go:117] "RemoveContainer" containerID="84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16"
Jul 16 20:02:23.941088 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.941067    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-974f7fcf5-62dcq"]
Jul 16 20:02:23.941160 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.941152    2579 scope.go:117] "RemoveContainer" containerID="84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16"
Jul 16 20:02:23.941538 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:02:23.941410    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16\": container with ID starting with 84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16 not found: ID does not exist" containerID="84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16"
Jul 16 20:02:23.941538 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.941458    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16"} err="failed to get container status \"84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16\": rpc error: code = NotFound desc = could not find container \"84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16\": container with ID starting with 84b0a218b325e8c4869b020efe124455d85e418ddb5908ea9d359d96cb879b16 not found: ID does not exist"
Jul 16 20:02:23.944761 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:23.944740    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-974f7fcf5-62dcq"]
Jul 16 20:02:24.403345 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:24.403269    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc4f8142-6d5e-4ba6-93db-171d28427222" path="/var/lib/kubelet/pods/dc4f8142-6d5e-4ba6-93db-171d28427222/volumes"
Jul 16 20:02:32.983105 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:32.983072    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:02:32.987786 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:32.987757    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-749644bcf7-f6frz"
Jul 16 20:02:33.955624 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:33.955579    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cd48f9b78-r96df" podUID="61644b21-5e63-4f64-a8fe-3742d61a8306" containerName="console" containerID="cri-o://cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3" gracePeriod=15
Jul 16 20:02:34.217690 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.217632    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:02:34.274079 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274045    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-oauth-serving-cert\") pod \"61644b21-5e63-4f64-a8fe-3742d61a8306\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") "
Jul 16 20:02:34.274243 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274106    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-service-ca\") pod \"61644b21-5e63-4f64-a8fe-3742d61a8306\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") "
Jul 16 20:02:34.274243 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274136    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-oauth-config\") pod \"61644b21-5e63-4f64-a8fe-3742d61a8306\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") "
Jul 16 20:02:34.274243 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274183    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-trusted-ca-bundle\") pod \"61644b21-5e63-4f64-a8fe-3742d61a8306\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") "
Jul 16 20:02:34.274243 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274223    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z87zj\" (UniqueName: \"kubernetes.io/projected/61644b21-5e63-4f64-a8fe-3742d61a8306-kube-api-access-z87zj\") pod \"61644b21-5e63-4f64-a8fe-3742d61a8306\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") "
Jul 16 20:02:34.274483 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274284    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-serving-cert\") pod \"61644b21-5e63-4f64-a8fe-3742d61a8306\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") "
Jul 16 20:02:34.274483 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274322    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-console-config\") pod \"61644b21-5e63-4f64-a8fe-3742d61a8306\" (UID: \"61644b21-5e63-4f64-a8fe-3742d61a8306\") "
Jul 16 20:02:34.274483 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274392    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "61644b21-5e63-4f64-a8fe-3742d61a8306" (UID: "61644b21-5e63-4f64-a8fe-3742d61a8306"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:02:34.274661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274512    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-service-ca" (OuterVolumeSpecName: "service-ca") pod "61644b21-5e63-4f64-a8fe-3742d61a8306" (UID: "61644b21-5e63-4f64-a8fe-3742d61a8306"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:02:34.274661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274556    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "61644b21-5e63-4f64-a8fe-3742d61a8306" (UID: "61644b21-5e63-4f64-a8fe-3742d61a8306"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:02:34.274661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274629    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-oauth-serving-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:34.274661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274651    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:34.274859 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.274818    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-console-config" (OuterVolumeSpecName: "console-config") pod "61644b21-5e63-4f64-a8fe-3742d61a8306" (UID: "61644b21-5e63-4f64-a8fe-3742d61a8306"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:02:34.276636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.276607    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "61644b21-5e63-4f64-a8fe-3742d61a8306" (UID: "61644b21-5e63-4f64-a8fe-3742d61a8306"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:02:34.276790 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.276764    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61644b21-5e63-4f64-a8fe-3742d61a8306-kube-api-access-z87zj" (OuterVolumeSpecName: "kube-api-access-z87zj") pod "61644b21-5e63-4f64-a8fe-3742d61a8306" (UID: "61644b21-5e63-4f64-a8fe-3742d61a8306"). InnerVolumeSpecName "kube-api-access-z87zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:02:34.276896 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.276874    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "61644b21-5e63-4f64-a8fe-3742d61a8306" (UID: "61644b21-5e63-4f64-a8fe-3742d61a8306"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:02:34.375720 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.375684    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z87zj\" (UniqueName: \"kubernetes.io/projected/61644b21-5e63-4f64-a8fe-3742d61a8306-kube-api-access-z87zj\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:34.375720 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.375719    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-serving-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:34.375927 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.375733    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-console-config\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:34.375927 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.375762    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61644b21-5e63-4f64-a8fe-3742d61a8306-console-oauth-config\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:34.375927 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.375775    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61644b21-5e63-4f64-a8fe-3742d61a8306-trusted-ca-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:02:34.965932 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.965891    2579 generic.go:356] "Generic (PLEG): container finished" podID="61644b21-5e63-4f64-a8fe-3742d61a8306" containerID="cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3" exitCode=0
Jul 16 20:02:34.966117 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.965957    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd48f9b78-r96df"
Jul 16 20:02:34.966117 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.965968    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd48f9b78-r96df" event={"ID":"61644b21-5e63-4f64-a8fe-3742d61a8306","Type":"ContainerDied","Data":"cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3"}
Jul 16 20:02:34.966117 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.966051    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd48f9b78-r96df" event={"ID":"61644b21-5e63-4f64-a8fe-3742d61a8306","Type":"ContainerDied","Data":"a8ca3c52cfdb0255198480d39cc821d86e875a9727ed3c8a0bf7edcfb88b7f43"}
Jul 16 20:02:34.966117 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.966073    2579 scope.go:117] "RemoveContainer" containerID="cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3"
Jul 16 20:02:34.974967 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.974931    2579 scope.go:117] "RemoveContainer" containerID="cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3"
Jul 16 20:02:34.975651 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:02:34.975569    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3\": container with ID starting with cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3 not found: ID does not exist" containerID="cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3"
Jul 16 20:02:34.975651 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.975619    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3"} err="failed to get container status \"cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3\": rpc error: code = NotFound desc = could not find container \"cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3\": container with ID starting with cd103ed601a0a382fb5792a8b7bb31d04dcd20f643b31eae47b05766a70c60d3 not found: ID does not exist"
Jul 16 20:02:34.982134 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.982104    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cd48f9b78-r96df"]
Jul 16 20:02:34.985797 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:34.985777    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cd48f9b78-r96df"]
Jul 16 20:02:36.403376 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:02:36.403344    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61644b21-5e63-4f64-a8fe-3742d61a8306" path="/var/lib/kubelet/pods/61644b21-5e63-4f64-a8fe-3742d61a8306/volumes"
Jul 16 20:03:04.951411 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.951376    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77c4cb7b55-wn9d2"]
Jul 16 20:03:04.951946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.951761    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61644b21-5e63-4f64-a8fe-3742d61a8306" containerName="console"
Jul 16 20:03:04.951946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.951777    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="61644b21-5e63-4f64-a8fe-3742d61a8306" containerName="console"
Jul 16 20:03:04.951946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.951794    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc4f8142-6d5e-4ba6-93db-171d28427222" containerName="console"
Jul 16 20:03:04.951946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.951799    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc4f8142-6d5e-4ba6-93db-171d28427222" containerName="console"
Jul 16 20:03:04.951946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.951857    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc4f8142-6d5e-4ba6-93db-171d28427222" containerName="console"
Jul 16 20:03:04.951946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.951866    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="61644b21-5e63-4f64-a8fe-3742d61a8306" containerName="console"
Jul 16 20:03:04.953959 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.953941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:04.956262 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.956239    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 20:03:04.957061 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.957043    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 20:03:04.957141 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.957130    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 20:03:04.957265 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.957246    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 20:03:04.957341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.957262    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 20:03:04.957341 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.957262    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 20:03:04.957491 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.957251    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 20:03:04.957676 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.957658    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z6795\""
Jul 16 20:03:04.964297 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.964277    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 16 20:03:04.970443 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:04.970410    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c4cb7b55-wn9d2"]
Jul 16 20:03:05.017655 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.017623    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-config\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.017838 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.017678    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-oauth-config\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.017838 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.017699    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-oauth-serving-cert\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.017838 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.017744    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-trusted-ca-bundle\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.017838 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.017767    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-service-ca\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.017838 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.017805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-serving-cert\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.018010 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.017841    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2ft8\" (UniqueName: \"kubernetes.io/projected/25dc3a5f-fecc-4e19-b1ad-d373849eebef-kube-api-access-l2ft8\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.118567 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.118531    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-serving-cert\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.118567 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.118572    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2ft8\" (UniqueName: \"kubernetes.io/projected/25dc3a5f-fecc-4e19-b1ad-d373849eebef-kube-api-access-l2ft8\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.118816 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.118595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-config\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.118816 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.118642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-oauth-config\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.118816 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.118660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-oauth-serving-cert\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.118816 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.118678    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-trusted-ca-bundle\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.118816 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.118697    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-service-ca\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.119506 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.119478    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-service-ca\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.119636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.119530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-oauth-serving-cert\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.119636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.119543    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-config\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.119753 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.119631    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-trusted-ca-bundle\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.121192 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.121164    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-oauth-config\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.121281 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.121262    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-serving-cert\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.132198 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.132167    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2ft8\" (UniqueName: \"kubernetes.io/projected/25dc3a5f-fecc-4e19-b1ad-d373849eebef-kube-api-access-l2ft8\") pod \"console-77c4cb7b55-wn9d2\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") " pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.263843 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.263757    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:05.387863 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:05.387702    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c4cb7b55-wn9d2"]
Jul 16 20:03:05.390562 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:03:05.390534    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25dc3a5f_fecc_4e19_b1ad_d373849eebef.slice/crio-e54a8a4f2d76037fa1e8cb6a6ec0f7868cf729d530e71f81e3667a9b3d8d61d3 WatchSource:0}: Error finding container e54a8a4f2d76037fa1e8cb6a6ec0f7868cf729d530e71f81e3667a9b3d8d61d3: Status 404 returned error can't find the container with id e54a8a4f2d76037fa1e8cb6a6ec0f7868cf729d530e71f81e3667a9b3d8d61d3
Jul 16 20:03:06.081347 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:06.081306    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c4cb7b55-wn9d2" event={"ID":"25dc3a5f-fecc-4e19-b1ad-d373849eebef","Type":"ContainerStarted","Data":"1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e"}
Jul 16 20:03:06.081347 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:06.081356    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c4cb7b55-wn9d2" event={"ID":"25dc3a5f-fecc-4e19-b1ad-d373849eebef","Type":"ContainerStarted","Data":"e54a8a4f2d76037fa1e8cb6a6ec0f7868cf729d530e71f81e3667a9b3d8d61d3"}
Jul 16 20:03:06.099007 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:06.098934    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77c4cb7b55-wn9d2" podStartSLOduration=2.098914249 podStartE2EDuration="2.098914249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:03:06.098870713 +0000 UTC m=+150.246966137" watchObservedRunningTime="2026-07-16 20:03:06.098914249 +0000 UTC m=+150.247009674"
Jul 16 20:03:08.436642 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:08.436609    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 20:03:08.437052 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:08.436912    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="alertmanager" containerID="cri-o://f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4" gracePeriod=120
Jul 16 20:03:08.437052 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:08.436975    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy-metric" containerID="cri-o://7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e" gracePeriod=120
Jul 16 20:03:08.437052 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:08.437005    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="prom-label-proxy" containerID="cri-o://97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc" gracePeriod=120
Jul 16 20:03:08.437221 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:08.437026    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy-web" containerID="cri-o://ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb" gracePeriod=120
Jul 16 20:03:08.437221 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:08.437041    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy" containerID="cri-o://f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e" gracePeriod=120
Jul 16 20:03:08.437221 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:08.437059    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="config-reloader" containerID="cri-o://462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3" gracePeriod=120
Jul 16 20:03:09.096723 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.096686    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerID="97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc" exitCode=0
Jul 16 20:03:09.096723 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.096711    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerID="f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e" exitCode=0
Jul 16 20:03:09.096723 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.096717    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerID="462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3" exitCode=0
Jul 16 20:03:09.096723 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.096723    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerID="f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4" exitCode=0
Jul 16 20:03:09.096975 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.096755    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerDied","Data":"97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc"}
Jul 16 20:03:09.096975 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.096820    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerDied","Data":"f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e"}
Jul 16 20:03:09.096975 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.096848    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerDied","Data":"462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3"}
Jul 16 20:03:09.096975 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.096874    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerDied","Data":"f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4"}
Jul 16 20:03:09.674642 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.674617    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:09.757804 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.757774    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-trusted-ca-bundle\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.757804 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.757807    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-metrics-client-ca\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.757827    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-cluster-tls-config\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.757860    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-main-tls\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.757886    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-web-config\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.757913    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hbzd\" (UniqueName: \"kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-kube-api-access-2hbzd\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.757934    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-web\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.757959    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.757986    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-tls-assets\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758392 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.758034    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-volume\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758392 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.758065    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-main-db\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758392 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.758104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-out\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758392 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.758150    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\" (UID: \"b2d098bd-ff6b-43f6-8299-60b27a454c8d\") "
Jul 16 20:03:09.758392 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.758208    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:03:09.758656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.758406    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-metrics-client-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.758656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.758476    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:03:09.759212 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.759182    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:03:09.760689 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.760614    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-kube-api-access-2hbzd" (OuterVolumeSpecName: "kube-api-access-2hbzd") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "kube-api-access-2hbzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:03:09.761381 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.761343    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:03:09.761498 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.761392    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-volume" (OuterVolumeSpecName: "config-volume") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:03:09.761498 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.761418    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:03:09.761644 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.761625    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:03:09.762046 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.762020    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:03:09.762274 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.762251    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-out" (OuterVolumeSpecName: "config-out") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:03:09.762565 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.762543    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:03:09.764846 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.764827    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:03:09.771077 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.771054    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-web-config" (OuterVolumeSpecName: "web-config") pod "b2d098bd-ff6b-43f6-8299-60b27a454c8d" (UID: "b2d098bd-ff6b-43f6-8299-60b27a454c8d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:03:09.859255 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859225    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859255 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859252    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859265    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-tls-assets\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859275    2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-volume\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859284    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-main-db\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859293    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2d098bd-ff6b-43f6-8299-60b27a454c8d-config-out\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859301    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859310    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2d098bd-ff6b-43f6-8299-60b27a454c8d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859319    2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-cluster-tls-config\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859327    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-secret-alertmanager-main-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859335    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2d098bd-ff6b-43f6-8299-60b27a454c8d-web-config\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:09.859410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:09.859343    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hbzd\" (UniqueName: \"kubernetes.io/projected/b2d098bd-ff6b-43f6-8299-60b27a454c8d-kube-api-access-2hbzd\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:03:10.103424 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.103330    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerID="7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e" exitCode=0
Jul 16 20:03:10.103424 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.103359    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerID="ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb" exitCode=0
Jul 16 20:03:10.103424 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.103402    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.103703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.103400    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerDied","Data":"7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e"}
Jul 16 20:03:10.103703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.103505    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerDied","Data":"ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb"}
Jul 16 20:03:10.103703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.103534    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b2d098bd-ff6b-43f6-8299-60b27a454c8d","Type":"ContainerDied","Data":"27c18b35e175883a1b90283a7e3d676963512dbe1c02af29a0cd986adc18f803"}
Jul 16 20:03:10.103703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.103572    2579 scope.go:117] "RemoveContainer" containerID="97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc"
Jul 16 20:03:10.111879 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.111863    2579 scope.go:117] "RemoveContainer" containerID="7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e"
Jul 16 20:03:10.118554 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.118539    2579 scope.go:117] "RemoveContainer" containerID="f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e"
Jul 16 20:03:10.126066 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.126042    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 20:03:10.128904 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.128882    2579 scope.go:117] "RemoveContainer" containerID="ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb"
Jul 16 20:03:10.129210 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.129192    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 20:03:10.135256 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.135238    2579 scope.go:117] "RemoveContainer" containerID="462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3"
Jul 16 20:03:10.141770 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.141752    2579 scope.go:117] "RemoveContainer" containerID="f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4"
Jul 16 20:03:10.148307 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.148291    2579 scope.go:117] "RemoveContainer" containerID="3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02"
Jul 16 20:03:10.154071 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154051    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 20:03:10.154376 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154359    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy-metric"
Jul 16 20:03:10.154461 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154379    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy-metric"
Jul 16 20:03:10.154461 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154400    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy"
Jul 16 20:03:10.154461 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154408    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy"
Jul 16 20:03:10.154461 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154421    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy-web"
Jul 16 20:03:10.154461 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154444    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy-web"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154464    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="alertmanager"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154473    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="alertmanager"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154485    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="init-config-reloader"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154493    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="init-config-reloader"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154508    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="prom-label-proxy"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154517    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="prom-label-proxy"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154528    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="config-reloader"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154536    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="config-reloader"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154607    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="prom-label-proxy"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154622    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="config-reloader"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154633    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="alertmanager"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154641    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy-metric"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154648    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy"
Jul 16 20:03:10.154739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154657    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" containerName="kube-rbac-proxy-web"
Jul 16 20:03:10.155377 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.154986    2579 scope.go:117] "RemoveContainer" containerID="97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc"
Jul 16 20:03:10.155377 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:03:10.155236    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc\": container with ID starting with 97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc not found: ID does not exist" containerID="97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc"
Jul 16 20:03:10.155377 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.155260    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc"} err="failed to get container status \"97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc\": rpc error: code = NotFound desc = could not find container \"97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc\": container with ID starting with 97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc not found: ID does not exist"
Jul 16 20:03:10.155377 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.155279    2579 scope.go:117] "RemoveContainer" containerID="7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e"
Jul 16 20:03:10.155657 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:03:10.155522    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e\": container with ID starting with 7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e not found: ID does not exist" containerID="7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e"
Jul 16 20:03:10.155657 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.155539    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e"} err="failed to get container status \"7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e\": rpc error: code = NotFound desc = could not find container \"7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e\": container with ID starting with 7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e not found: ID does not exist"
Jul 16 20:03:10.155657 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.155552    2579 scope.go:117] "RemoveContainer" containerID="f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e"
Jul 16 20:03:10.155796 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:03:10.155779    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e\": container with ID starting with f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e not found: ID does not exist" containerID="f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e"
Jul 16 20:03:10.155845 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.155799    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e"} err="failed to get container status \"f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e\": rpc error: code = NotFound desc = could not find container \"f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e\": container with ID starting with f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e not found: ID does not exist"
Jul 16 20:03:10.155845 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.155813    2579 scope.go:117] "RemoveContainer" containerID="ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb"
Jul 16 20:03:10.156038 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:03:10.156017    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb\": container with ID starting with ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb not found: ID does not exist" containerID="ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb"
Jul 16 20:03:10.156096 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.156041    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb"} err="failed to get container status \"ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb\": rpc error: code = NotFound desc = could not find container \"ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb\": container with ID starting with ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb not found: ID does not exist"
Jul 16 20:03:10.156096 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.156068    2579 scope.go:117] "RemoveContainer" containerID="462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3"
Jul 16 20:03:10.156360 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:03:10.156343    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3\": container with ID starting with 462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3 not found: ID does not exist" containerID="462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3"
Jul 16 20:03:10.156503 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.156362    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3"} err="failed to get container status \"462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3\": rpc error: code = NotFound desc = could not find container \"462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3\": container with ID starting with 462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3 not found: ID does not exist"
Jul 16 20:03:10.156503 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.156375    2579 scope.go:117] "RemoveContainer" containerID="f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4"
Jul 16 20:03:10.156648 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:03:10.156630    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4\": container with ID starting with f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4 not found: ID does not exist" containerID="f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4"
Jul 16 20:03:10.156707 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.156689    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4"} err="failed to get container status \"f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4\": rpc error: code = NotFound desc = could not find container \"f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4\": container with ID starting with f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4 not found: ID does not exist"
Jul 16 20:03:10.156741 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.156708    2579 scope.go:117] "RemoveContainer" containerID="3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02"
Jul 16 20:03:10.156945 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:03:10.156929    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02\": container with ID starting with 3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02 not found: ID does not exist" containerID="3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02"
Jul 16 20:03:10.156991 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.156952    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02"} err="failed to get container status \"3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02\": rpc error: code = NotFound desc = could not find container \"3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02\": container with ID starting with 3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02 not found: ID does not exist"
Jul 16 20:03:10.156991 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.156967    2579 scope.go:117] "RemoveContainer" containerID="97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc"
Jul 16 20:03:10.157179 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.157163    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc"} err="failed to get container status \"97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc\": rpc error: code = NotFound desc = could not find container \"97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc\": container with ID starting with 97590195de6f4298c8aecda98b30c99ea559b30c4e3eff080221812eea8424fc not found: ID does not exist"
Jul 16 20:03:10.157220 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.157179    2579 scope.go:117] "RemoveContainer" containerID="7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e"
Jul 16 20:03:10.157374 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.157358    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e"} err="failed to get container status \"7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e\": rpc error: code = NotFound desc = could not find container \"7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e\": container with ID starting with 7f7816c51c7d742b64e26a8ea653c4367a0d9b972ebbd6d3e86617b728d0925e not found: ID does not exist"
Jul 16 20:03:10.157411 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.157375    2579 scope.go:117] "RemoveContainer" containerID="f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e"
Jul 16 20:03:10.157570 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.157553    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e"} err="failed to get container status \"f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e\": rpc error: code = NotFound desc = could not find container \"f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e\": container with ID starting with f39aa6d65775482e673190c46d08c15ab30fcc0f3cf97afbc180e69aa71c565e not found: ID does not exist"
Jul 16 20:03:10.157622 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.157570    2579 scope.go:117] "RemoveContainer" containerID="ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb"
Jul 16 20:03:10.157744 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.157731    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb"} err="failed to get container status \"ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb\": rpc error: code = NotFound desc = could not find container \"ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb\": container with ID starting with ca8d98a317ea081e853dcda75989e1622a06c65a042ff1783a2e80d328c87bcb not found: ID does not exist"
Jul 16 20:03:10.157778 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.157744    2579 scope.go:117] "RemoveContainer" containerID="462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3"
Jul 16 20:03:10.157946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.157928    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3"} err="failed to get container status \"462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3\": rpc error: code = NotFound desc = could not find container \"462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3\": container with ID starting with 462a0a7754600158d18ab124f596cff30210d88a1cb404cc8f302808660613b3 not found: ID does not exist"
Jul 16 20:03:10.157995 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.157946    2579 scope.go:117] "RemoveContainer" containerID="f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4"
Jul 16 20:03:10.158112 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.158096    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4"} err="failed to get container status \"f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4\": rpc error: code = NotFound desc = could not find container \"f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4\": container with ID starting with f0cb323a942f7bcbe2af8e395a9bf2411e01127ea38e7cb55058b80a0b593de4 not found: ID does not exist"
Jul 16 20:03:10.158167 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.158113    2579 scope.go:117] "RemoveContainer" containerID="3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02"
Jul 16 20:03:10.158265 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.158250    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02"} err="failed to get container status \"3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02\": rpc error: code = NotFound desc = could not find container \"3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02\": container with ID starting with 3073bad28556d27ee5e280a46eb826ed57741f87e042ba9c187d91d580567a02 not found: ID does not exist"
Jul 16 20:03:10.158579 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.158563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.160779 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.160759    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 16 20:03:10.160862 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.160801    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 16 20:03:10.160999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.160980    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 16 20:03:10.161197 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.161182    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 16 20:03:10.161274 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.161185    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8gnxc\""
Jul 16 20:03:10.161274 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.161185    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 16 20:03:10.161390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.161340    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 16 20:03:10.161678 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.161660    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 16 20:03:10.164012 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.162100    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 16 20:03:10.170333 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.170302    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 16 20:03:10.172357 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.172337    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 20:03:10.262494 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262494 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-config-volume\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262614    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-web-config\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmz96\" (UniqueName: \"kubernetes.io/projected/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-kube-api-access-zmz96\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262749    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.262785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.262775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-config-out\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.363967 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.363889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.363967 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.363931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.363967 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.363965    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.364193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.363989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-config-volume\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.364193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.364007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.364193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.364022    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-web-config\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.364193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.364038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.364193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.364061    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.364193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.364085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmz96\" (UniqueName: \"kubernetes.io/projected/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-kube-api-access-zmz96\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.364193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.364111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.364193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.364142    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.364193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.364163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.364193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.364195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-config-out\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.365030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.364899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.367197 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.367042    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.367197 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.367069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.367197 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.367079    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-config-volume\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.367197 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.367071    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-config-out\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.367197 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.367120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.367477 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.367247    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-web-config\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.367477 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.367329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.367477 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.367358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.367602 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.367488    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.367863 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.367845    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.369052 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.369036    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.376576 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.376559    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmz96\" (UniqueName: \"kubernetes.io/projected/4227dd95-01d9-4cfd-9a7b-e0e1e22a824d-kube-api-access-zmz96\") pod \"alertmanager-main-0\" (UID: \"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.402286 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.402261    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d098bd-ff6b-43f6-8299-60b27a454c8d" path="/var/lib/kubelet/pods/b2d098bd-ff6b-43f6-8299-60b27a454c8d/volumes"
Jul 16 20:03:10.471963 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.471932    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 20:03:10.611038 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:10.611013    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 20:03:10.613177 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:03:10.613148    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4227dd95_01d9_4cfd_9a7b_e0e1e22a824d.slice/crio-6b7e8e01c96e84bee7d9fb476f0a9ee2d361311d2245e595183577c136841a34 WatchSource:0}: Error finding container 6b7e8e01c96e84bee7d9fb476f0a9ee2d361311d2245e595183577c136841a34: Status 404 returned error can't find the container with id 6b7e8e01c96e84bee7d9fb476f0a9ee2d361311d2245e595183577c136841a34
Jul 16 20:03:11.109840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:11.109804    2579 generic.go:356] "Generic (PLEG): container finished" podID="4227dd95-01d9-4cfd-9a7b-e0e1e22a824d" containerID="1a50242c7fd6f090f41031f8791f3216fe687e39d98bd378cd1dfffd2b0bfc1a" exitCode=0
Jul 16 20:03:11.110325 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:11.109872    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d","Type":"ContainerDied","Data":"1a50242c7fd6f090f41031f8791f3216fe687e39d98bd378cd1dfffd2b0bfc1a"}
Jul 16 20:03:11.110325 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:11.109945    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d","Type":"ContainerStarted","Data":"6b7e8e01c96e84bee7d9fb476f0a9ee2d361311d2245e595183577c136841a34"}
Jul 16 20:03:12.122145 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:12.122111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d","Type":"ContainerStarted","Data":"a0062048cd2f10096921a5389cf60b1783e4a86deef63c5b14e1b05efc92cf4e"}
Jul 16 20:03:12.122522 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:12.122206    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d","Type":"ContainerStarted","Data":"30d89e4162657d5ffc0e7600e94163645db78aac833395c0896d42df05dbeff3"}
Jul 16 20:03:12.122522 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:12.122241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d","Type":"ContainerStarted","Data":"3f1aa4f1351355dc66de26e6d021c39f36ba1de018bb5b2936521ed5254237aa"}
Jul 16 20:03:12.122522 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:12.122265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d","Type":"ContainerStarted","Data":"53fcdebe7d3e94c4df033e43ee7791298ccd0759e8447a8d7add0563704c026e"}
Jul 16 20:03:12.122522 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:12.122296    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d","Type":"ContainerStarted","Data":"1e23b4091cc085abbb082f8fce12504e0c680f3db58bea6152a1e171eafda367"}
Jul 16 20:03:12.122522 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:12.122321    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4227dd95-01d9-4cfd-9a7b-e0e1e22a824d","Type":"ContainerStarted","Data":"d4f8a59477f546c1ec86bb34f10635898ab78853fb49d86532bd9add758dec94"}
Jul 16 20:03:12.144780 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:12.144733    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.129101288 podStartE2EDuration="2.144717228s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.61594ms" isStatefulPod=false podCreationTimestamp="2026-07-16 20:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:03:12.143146304 +0000 UTC m=+156.291241766" watchObservedRunningTime="2026-07-16 20:03:12.144717228 +0000 UTC m=+156.292812694"
Jul 16 20:03:15.264281 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:15.264244    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:15.264702 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:15.264304    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:15.269036 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:15.269012    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:03:16.140887 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:03:16.140855    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:04:40.962242 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:40.962204    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"]
Jul 16 20:04:40.964809 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:40.964790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:40.967223 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:40.967191    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g8prd\""
Jul 16 20:04:40.967357 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:40.967295    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 16 20:04:40.967357 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:40.967295    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 16 20:04:40.978156 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:40.978126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"]
Jul 16 20:04:41.035774 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.035738    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:41.035774 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.035779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:41.036042 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.035907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mjg6\" (UniqueName: \"kubernetes.io/projected/d52c267c-3139-4147-be75-6eaf2d32c22e-kube-api-access-5mjg6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:41.137271 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.137224    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mjg6\" (UniqueName: \"kubernetes.io/projected/d52c267c-3139-4147-be75-6eaf2d32c22e-kube-api-access-5mjg6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:41.137506 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.137288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:41.137506 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.137307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:41.137717 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.137699    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:41.137768 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.137735    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:41.146210 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.146183    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mjg6\" (UniqueName: \"kubernetes.io/projected/d52c267c-3139-4147-be75-6eaf2d32c22e-kube-api-access-5mjg6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:41.273561 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.273467    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:04:41.404639 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.404594    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"]
Jul 16 20:04:41.407316 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:04:41.407288    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52c267c_3139_4147_be75_6eaf2d32c22e.slice/crio-9255549c6c0a15442824d8640b2684b03621607c94f93ca940537010562ce0f7 WatchSource:0}: Error finding container 9255549c6c0a15442824d8640b2684b03621607c94f93ca940537010562ce0f7: Status 404 returned error can't find the container with id 9255549c6c0a15442824d8640b2684b03621607c94f93ca940537010562ce0f7
Jul 16 20:04:41.455525 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:41.455487    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd" event={"ID":"d52c267c-3139-4147-be75-6eaf2d32c22e","Type":"ContainerStarted","Data":"9255549c6c0a15442824d8640b2684b03621607c94f93ca940537010562ce0f7"}
Jul 16 20:04:50.494383 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:50.494346    2579 generic.go:356] "Generic (PLEG): container finished" podID="d52c267c-3139-4147-be75-6eaf2d32c22e" containerID="d9cb3214688a98ca5a0f15c74f32e5c7fe1aec30ede5de8c73f62a6d5742a8f8" exitCode=0
Jul 16 20:04:50.494855 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:50.494448    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd" event={"ID":"d52c267c-3139-4147-be75-6eaf2d32c22e","Type":"ContainerDied","Data":"d9cb3214688a98ca5a0f15c74f32e5c7fe1aec30ede5de8c73f62a6d5742a8f8"}
Jul 16 20:04:53.507709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:53.507672    2579 generic.go:356] "Generic (PLEG): container finished" podID="d52c267c-3139-4147-be75-6eaf2d32c22e" containerID="f386f640897fe3e460f456a9e9dbffbb2957096b71321b00e082950df580d165" exitCode=0
Jul 16 20:04:53.508115 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:04:53.507764    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd" event={"ID":"d52c267c-3139-4147-be75-6eaf2d32c22e","Type":"ContainerDied","Data":"f386f640897fe3e460f456a9e9dbffbb2957096b71321b00e082950df580d165"}
Jul 16 20:05:03.550566 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:03.550532    2579 generic.go:356] "Generic (PLEG): container finished" podID="d52c267c-3139-4147-be75-6eaf2d32c22e" containerID="1b5fe17400f395eeab8bc983256ffc4a0e6ed906728d367cdbe586b5fb24230a" exitCode=0
Jul 16 20:05:03.550962 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:03.550615    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd" event={"ID":"d52c267c-3139-4147-be75-6eaf2d32c22e","Type":"ContainerDied","Data":"1b5fe17400f395eeab8bc983256ffc4a0e6ed906728d367cdbe586b5fb24230a"}
Jul 16 20:05:04.723683 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:04.723648    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:05:04.845283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:04.845185    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-util\") pod \"d52c267c-3139-4147-be75-6eaf2d32c22e\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") "
Jul 16 20:05:04.845283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:04.845240    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-bundle\") pod \"d52c267c-3139-4147-be75-6eaf2d32c22e\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") "
Jul 16 20:05:04.845500 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:04.845295    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mjg6\" (UniqueName: \"kubernetes.io/projected/d52c267c-3139-4147-be75-6eaf2d32c22e-kube-api-access-5mjg6\") pod \"d52c267c-3139-4147-be75-6eaf2d32c22e\" (UID: \"d52c267c-3139-4147-be75-6eaf2d32c22e\") "
Jul 16 20:05:04.846015 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:04.845990    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-bundle" (OuterVolumeSpecName: "bundle") pod "d52c267c-3139-4147-be75-6eaf2d32c22e" (UID: "d52c267c-3139-4147-be75-6eaf2d32c22e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:05:04.847584 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:04.847551    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d52c267c-3139-4147-be75-6eaf2d32c22e-kube-api-access-5mjg6" (OuterVolumeSpecName: "kube-api-access-5mjg6") pod "d52c267c-3139-4147-be75-6eaf2d32c22e" (UID: "d52c267c-3139-4147-be75-6eaf2d32c22e"). InnerVolumeSpecName "kube-api-access-5mjg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:05:04.850510 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:04.850485    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-util" (OuterVolumeSpecName: "util") pod "d52c267c-3139-4147-be75-6eaf2d32c22e" (UID: "d52c267c-3139-4147-be75-6eaf2d32c22e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:05:04.946196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:04.946154    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mjg6\" (UniqueName: \"kubernetes.io/projected/d52c267c-3139-4147-be75-6eaf2d32c22e-kube-api-access-5mjg6\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:05:04.946196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:04.946188    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:05:04.946196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:04.946199    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d52c267c-3139-4147-be75-6eaf2d32c22e-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:05:05.559542 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:05.559512    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd"
Jul 16 20:05:05.559708 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:05.559505    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzzfd" event={"ID":"d52c267c-3139-4147-be75-6eaf2d32c22e","Type":"ContainerDied","Data":"9255549c6c0a15442824d8640b2684b03621607c94f93ca940537010562ce0f7"}
Jul 16 20:05:05.559708 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:05.559630    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9255549c6c0a15442824d8640b2684b03621607c94f93ca940537010562ce0f7"
Jul 16 20:05:09.151521 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.151483    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"]
Jul 16 20:05:09.152706 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.152681    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d52c267c-3139-4147-be75-6eaf2d32c22e" containerName="pull"
Jul 16 20:05:09.152869 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.152857    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d52c267c-3139-4147-be75-6eaf2d32c22e" containerName="pull"
Jul 16 20:05:09.152975 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.152964    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d52c267c-3139-4147-be75-6eaf2d32c22e" containerName="extract"
Jul 16 20:05:09.153053 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.153042    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d52c267c-3139-4147-be75-6eaf2d32c22e" containerName="extract"
Jul 16 20:05:09.153160 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.153149    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d52c267c-3139-4147-be75-6eaf2d32c22e" containerName="util"
Jul 16 20:05:09.153233 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.153224    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d52c267c-3139-4147-be75-6eaf2d32c22e" containerName="util"
Jul 16 20:05:09.153451 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.153421    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d52c267c-3139-4147-be75-6eaf2d32c22e" containerName="extract"
Jul 16 20:05:09.156673 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.156644    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"
Jul 16 20:05:09.159456 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.159399    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:05:09.159456 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.159408    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 16 20:05:09.159661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.159641    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mb6dn\""
Jul 16 20:05:09.167248 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.167221    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"]
Jul 16 20:05:09.282032 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.281996    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef1e19c3-fe48-4efd-ab8d-38121cad1e7e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-k8mpq\" (UID: \"ef1e19c3-fe48-4efd-ab8d-38121cad1e7e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"
Jul 16 20:05:09.282224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.282088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxcpc\" (UniqueName: \"kubernetes.io/projected/ef1e19c3-fe48-4efd-ab8d-38121cad1e7e-kube-api-access-rxcpc\") pod \"cert-manager-operator-controller-manager-55c444bf86-k8mpq\" (UID: \"ef1e19c3-fe48-4efd-ab8d-38121cad1e7e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"
Jul 16 20:05:09.382792 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.382756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxcpc\" (UniqueName: \"kubernetes.io/projected/ef1e19c3-fe48-4efd-ab8d-38121cad1e7e-kube-api-access-rxcpc\") pod \"cert-manager-operator-controller-manager-55c444bf86-k8mpq\" (UID: \"ef1e19c3-fe48-4efd-ab8d-38121cad1e7e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"
Jul 16 20:05:09.383000 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.382835    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef1e19c3-fe48-4efd-ab8d-38121cad1e7e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-k8mpq\" (UID: \"ef1e19c3-fe48-4efd-ab8d-38121cad1e7e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"
Jul 16 20:05:09.383243 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.383221    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef1e19c3-fe48-4efd-ab8d-38121cad1e7e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-k8mpq\" (UID: \"ef1e19c3-fe48-4efd-ab8d-38121cad1e7e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"
Jul 16 20:05:09.392107 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.392080    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxcpc\" (UniqueName: \"kubernetes.io/projected/ef1e19c3-fe48-4efd-ab8d-38121cad1e7e-kube-api-access-rxcpc\") pod \"cert-manager-operator-controller-manager-55c444bf86-k8mpq\" (UID: \"ef1e19c3-fe48-4efd-ab8d-38121cad1e7e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"
Jul 16 20:05:09.469486 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.469452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"
Jul 16 20:05:09.599465 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:09.599423    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq"]
Jul 16 20:05:09.601969 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:05:09.601940    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef1e19c3_fe48_4efd_ab8d_38121cad1e7e.slice/crio-89980238d3e7d03e9730522b09418fe698dc2d529be3a8e2da3e6722941e5c96 WatchSource:0}: Error finding container 89980238d3e7d03e9730522b09418fe698dc2d529be3a8e2da3e6722941e5c96: Status 404 returned error can't find the container with id 89980238d3e7d03e9730522b09418fe698dc2d529be3a8e2da3e6722941e5c96
Jul 16 20:05:10.581945 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:10.581912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq" event={"ID":"ef1e19c3-fe48-4efd-ab8d-38121cad1e7e","Type":"ContainerStarted","Data":"89980238d3e7d03e9730522b09418fe698dc2d529be3a8e2da3e6722941e5c96"}
Jul 16 20:05:11.588335 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:11.588244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq" event={"ID":"ef1e19c3-fe48-4efd-ab8d-38121cad1e7e","Type":"ContainerStarted","Data":"c35ad32e8263de5f1a7853b3340f650c85642b2e01fc37a20b227a5d9996e6fe"}
Jul 16 20:05:11.611518 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:11.611458    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k8mpq" podStartSLOduration=1.038156019 podStartE2EDuration="2.611421124s" totalImagesPullingTime="1.573265105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:05:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:05:11.610774105 +0000 UTC m=+275.758869532" watchObservedRunningTime="2026-07-16 20:05:11.611421124 +0000 UTC m=+275.759516549"
Jul 16 20:05:12.866653 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:12.866618    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"]
Jul 16 20:05:12.868921 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:12.868901    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:12.871523 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:12.871495    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 16 20:05:12.871523 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:12.871518    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 16 20:05:12.872243 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:12.872225    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g8prd\""
Jul 16 20:05:12.881291 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:12.881269    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"]
Jul 16 20:05:13.015211 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.015177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:13.015374 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.015232    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqqst\" (UniqueName: \"kubernetes.io/projected/218504c1-39ef-4a70-8f09-a538a3ca27d7-kube-api-access-kqqst\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:13.015374 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.015298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:13.116283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.116227    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqqst\" (UniqueName: \"kubernetes.io/projected/218504c1-39ef-4a70-8f09-a538a3ca27d7-kube-api-access-kqqst\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:13.116469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.116387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:13.116527 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.116479    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:13.116785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.116746    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:13.116785 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.116773    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:13.126948 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.126928    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqqst\" (UniqueName: \"kubernetes.io/projected/218504c1-39ef-4a70-8f09-a538a3ca27d7-kube-api-access-kqqst\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:13.178004 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.177981    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:13.303409 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.303378    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"]
Jul 16 20:05:13.306832 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:05:13.306806    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod218504c1_39ef_4a70_8f09_a538a3ca27d7.slice/crio-fc22184a874fd71811a34ca68005c85b031f3f2f772fdc82444affd0320baee9 WatchSource:0}: Error finding container fc22184a874fd71811a34ca68005c85b031f3f2f772fdc82444affd0320baee9: Status 404 returned error can't find the container with id fc22184a874fd71811a34ca68005c85b031f3f2f772fdc82444affd0320baee9
Jul 16 20:05:13.602007 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.601963    2579 generic.go:356] "Generic (PLEG): container finished" podID="218504c1-39ef-4a70-8f09-a538a3ca27d7" containerID="22e8ef22acafee4a3ab44f6b76be2c058f9e2c5150da901af1e2550ecd4b0f16" exitCode=0
Jul 16 20:05:13.602163 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.602025    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq" event={"ID":"218504c1-39ef-4a70-8f09-a538a3ca27d7","Type":"ContainerDied","Data":"22e8ef22acafee4a3ab44f6b76be2c058f9e2c5150da901af1e2550ecd4b0f16"}
Jul 16 20:05:13.602163 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.602114    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq" event={"ID":"218504c1-39ef-4a70-8f09-a538a3ca27d7","Type":"ContainerStarted","Data":"fc22184a874fd71811a34ca68005c85b031f3f2f772fdc82444affd0320baee9"}
Jul 16 20:05:13.959687 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.959653    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9dwjt"]
Jul 16 20:05:13.962257 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.962236    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt"
Jul 16 20:05:13.964623 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.964602    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 16 20:05:13.964721 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.964652    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 16 20:05:13.964786 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.964772    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qp7zv\""
Jul 16 20:05:13.972862 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:13.972838    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9dwjt"]
Jul 16 20:05:14.123598 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:14.123567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/180df590-aca2-4ebb-a479-e7a648794138-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9dwjt\" (UID: \"180df590-aca2-4ebb-a479-e7a648794138\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt"
Jul 16 20:05:14.123766 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:14.123630    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6rq6\" (UniqueName: \"kubernetes.io/projected/180df590-aca2-4ebb-a479-e7a648794138-kube-api-access-k6rq6\") pod \"cert-manager-webhook-6486ffd997-9dwjt\" (UID: \"180df590-aca2-4ebb-a479-e7a648794138\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt"
Jul 16 20:05:14.224817 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:14.224727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/180df590-aca2-4ebb-a479-e7a648794138-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9dwjt\" (UID: \"180df590-aca2-4ebb-a479-e7a648794138\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt"
Jul 16 20:05:14.224987 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:14.224852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6rq6\" (UniqueName: \"kubernetes.io/projected/180df590-aca2-4ebb-a479-e7a648794138-kube-api-access-k6rq6\") pod \"cert-manager-webhook-6486ffd997-9dwjt\" (UID: \"180df590-aca2-4ebb-a479-e7a648794138\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt"
Jul 16 20:05:14.233399 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:14.233369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/180df590-aca2-4ebb-a479-e7a648794138-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9dwjt\" (UID: \"180df590-aca2-4ebb-a479-e7a648794138\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt"
Jul 16 20:05:14.233604 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:14.233534    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6rq6\" (UniqueName: \"kubernetes.io/projected/180df590-aca2-4ebb-a479-e7a648794138-kube-api-access-k6rq6\") pod \"cert-manager-webhook-6486ffd997-9dwjt\" (UID: \"180df590-aca2-4ebb-a479-e7a648794138\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt"
Jul 16 20:05:14.281345 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:14.281316    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt"
Jul 16 20:05:14.430914 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:14.430891    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9dwjt"]
Jul 16 20:05:14.433594 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:05:14.433564    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod180df590_aca2_4ebb_a479_e7a648794138.slice/crio-853f88078e584ed818f88bddfecf885e7e8ffec12069b7e054c4bcc3c8a65a1f WatchSource:0}: Error finding container 853f88078e584ed818f88bddfecf885e7e8ffec12069b7e054c4bcc3c8a65a1f: Status 404 returned error can't find the container with id 853f88078e584ed818f88bddfecf885e7e8ffec12069b7e054c4bcc3c8a65a1f
Jul 16 20:05:14.608821 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:14.608731    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt" event={"ID":"180df590-aca2-4ebb-a479-e7a648794138","Type":"ContainerStarted","Data":"853f88078e584ed818f88bddfecf885e7e8ffec12069b7e054c4bcc3c8a65a1f"}
Jul 16 20:05:15.617235 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:15.617198    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq" event={"ID":"218504c1-39ef-4a70-8f09-a538a3ca27d7","Type":"ContainerStarted","Data":"46e10b6f7558a4b39de8a9b4b4744085a7cc78913977d0c16bfc1cefada54900"}
Jul 16 20:05:16.623472 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:16.623408    2579 generic.go:356] "Generic (PLEG): container finished" podID="218504c1-39ef-4a70-8f09-a538a3ca27d7" containerID="46e10b6f7558a4b39de8a9b4b4744085a7cc78913977d0c16bfc1cefada54900" exitCode=0
Jul 16 20:05:16.623925 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:16.623479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq" event={"ID":"218504c1-39ef-4a70-8f09-a538a3ca27d7","Type":"ContainerDied","Data":"46e10b6f7558a4b39de8a9b4b4744085a7cc78913977d0c16bfc1cefada54900"}
Jul 16 20:05:17.629652 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:17.629615    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt" event={"ID":"180df590-aca2-4ebb-a479-e7a648794138","Type":"ContainerStarted","Data":"cb28fe015d989028a048c68ecaca36db3c339ba31efcdb8e695149af10deb222"}
Jul 16 20:05:17.630023 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:17.629700    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt"
Jul 16 20:05:17.631649 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:17.631628    2579 generic.go:356] "Generic (PLEG): container finished" podID="218504c1-39ef-4a70-8f09-a538a3ca27d7" containerID="967ed25780f1d31bc9b401c82d9f0c3b06649c4535d2764abaf9267f585249d8" exitCode=0
Jul 16 20:05:17.631730 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:17.631664    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq" event={"ID":"218504c1-39ef-4a70-8f09-a538a3ca27d7","Type":"ContainerDied","Data":"967ed25780f1d31bc9b401c82d9f0c3b06649c4535d2764abaf9267f585249d8"}
Jul 16 20:05:17.647669 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:17.647621    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt" podStartSLOduration=1.624380581 podStartE2EDuration="4.647609285s" totalImagesPullingTime="3.023228704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:05:17.64687116 +0000 UTC m=+281.794966584" watchObservedRunningTime="2026-07-16 20:05:17.647609285 +0000 UTC m=+281.795704707"
Jul 16 20:05:18.759206 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:18.759179    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:18.862817 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:18.862788    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-util\") pod \"218504c1-39ef-4a70-8f09-a538a3ca27d7\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") "
Jul 16 20:05:18.862817 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:18.862825    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqqst\" (UniqueName: \"kubernetes.io/projected/218504c1-39ef-4a70-8f09-a538a3ca27d7-kube-api-access-kqqst\") pod \"218504c1-39ef-4a70-8f09-a538a3ca27d7\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") "
Jul 16 20:05:18.863023 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:18.862865    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-bundle\") pod \"218504c1-39ef-4a70-8f09-a538a3ca27d7\" (UID: \"218504c1-39ef-4a70-8f09-a538a3ca27d7\") "
Jul 16 20:05:18.863296 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:18.863268    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-bundle" (OuterVolumeSpecName: "bundle") pod "218504c1-39ef-4a70-8f09-a538a3ca27d7" (UID: "218504c1-39ef-4a70-8f09-a538a3ca27d7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:05:18.864976 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:18.864952    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/218504c1-39ef-4a70-8f09-a538a3ca27d7-kube-api-access-kqqst" (OuterVolumeSpecName: "kube-api-access-kqqst") pod "218504c1-39ef-4a70-8f09-a538a3ca27d7" (UID: "218504c1-39ef-4a70-8f09-a538a3ca27d7"). InnerVolumeSpecName "kube-api-access-kqqst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:05:18.867774 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:18.867753    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-util" (OuterVolumeSpecName: "util") pod "218504c1-39ef-4a70-8f09-a538a3ca27d7" (UID: "218504c1-39ef-4a70-8f09-a538a3ca27d7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:05:18.964009 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:18.963966    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:05:18.964009 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:18.964006    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqqst\" (UniqueName: \"kubernetes.io/projected/218504c1-39ef-4a70-8f09-a538a3ca27d7-kube-api-access-kqqst\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:05:18.964193 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:18.964025    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/218504c1-39ef-4a70-8f09-a538a3ca27d7-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:05:19.641695 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:19.641665    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq"
Jul 16 20:05:19.641695 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:19.641680    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkmtcq" event={"ID":"218504c1-39ef-4a70-8f09-a538a3ca27d7","Type":"ContainerDied","Data":"fc22184a874fd71811a34ca68005c85b031f3f2f772fdc82444affd0320baee9"}
Jul 16 20:05:19.641894 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:19.641731    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc22184a874fd71811a34ca68005c85b031f3f2f772fdc82444affd0320baee9"
Jul 16 20:05:21.164180 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.164137    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"]
Jul 16 20:05:21.164781 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.164610    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="218504c1-39ef-4a70-8f09-a538a3ca27d7" containerName="extract"
Jul 16 20:05:21.164781 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.164630    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="218504c1-39ef-4a70-8f09-a538a3ca27d7" containerName="extract"
Jul 16 20:05:21.164781 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.164647    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="218504c1-39ef-4a70-8f09-a538a3ca27d7" containerName="pull"
Jul 16 20:05:21.164781 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.164656    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="218504c1-39ef-4a70-8f09-a538a3ca27d7" containerName="pull"
Jul 16 20:05:21.164781 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.164676    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="218504c1-39ef-4a70-8f09-a538a3ca27d7" containerName="util"
Jul 16 20:05:21.164781 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.164686    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="218504c1-39ef-4a70-8f09-a538a3ca27d7" containerName="util"
Jul 16 20:05:21.164781 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.164759    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="218504c1-39ef-4a70-8f09-a538a3ca27d7" containerName="extract"
Jul 16 20:05:21.166870 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.166848    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"
Jul 16 20:05:21.168867 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.168849    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-97vq7\""
Jul 16 20:05:21.174821 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.174800    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"]
Jul 16 20:05:21.280636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.280603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6604e787-06da-439a-ab8a-50bf84278da6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n2bdv\" (UID: \"6604e787-06da-439a-ab8a-50bf84278da6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"
Jul 16 20:05:21.280799 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.280682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqtv5\" (UniqueName: \"kubernetes.io/projected/6604e787-06da-439a-ab8a-50bf84278da6-kube-api-access-dqtv5\") pod \"cert-manager-cainjector-559cdd8c87-n2bdv\" (UID: \"6604e787-06da-439a-ab8a-50bf84278da6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"
Jul 16 20:05:21.382009 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.381978    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqtv5\" (UniqueName: \"kubernetes.io/projected/6604e787-06da-439a-ab8a-50bf84278da6-kube-api-access-dqtv5\") pod \"cert-manager-cainjector-559cdd8c87-n2bdv\" (UID: \"6604e787-06da-439a-ab8a-50bf84278da6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"
Jul 16 20:05:21.382177 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.382033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6604e787-06da-439a-ab8a-50bf84278da6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n2bdv\" (UID: \"6604e787-06da-439a-ab8a-50bf84278da6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"
Jul 16 20:05:21.389722 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.389694    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6604e787-06da-439a-ab8a-50bf84278da6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n2bdv\" (UID: \"6604e787-06da-439a-ab8a-50bf84278da6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"
Jul 16 20:05:21.389837 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.389791    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqtv5\" (UniqueName: \"kubernetes.io/projected/6604e787-06da-439a-ab8a-50bf84278da6-kube-api-access-dqtv5\") pod \"cert-manager-cainjector-559cdd8c87-n2bdv\" (UID: \"6604e787-06da-439a-ab8a-50bf84278da6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"
Jul 16 20:05:21.476216 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.476146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"
Jul 16 20:05:21.599887 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.599859    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv"]
Jul 16 20:05:21.602714 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:05:21.602690    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6604e787_06da_439a_ab8a_50bf84278da6.slice/crio-7f7bcc0e369e7b28ee6aa01b3ff826ab2d6cb77ad3898d3d40787b1a8e287b14 WatchSource:0}: Error finding container 7f7bcc0e369e7b28ee6aa01b3ff826ab2d6cb77ad3898d3d40787b1a8e287b14: Status 404 returned error can't find the container with id 7f7bcc0e369e7b28ee6aa01b3ff826ab2d6cb77ad3898d3d40787b1a8e287b14
Jul 16 20:05:21.651207 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:21.651181    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv" event={"ID":"6604e787-06da-439a-ab8a-50bf84278da6","Type":"ContainerStarted","Data":"7f7bcc0e369e7b28ee6aa01b3ff826ab2d6cb77ad3898d3d40787b1a8e287b14"}
Jul 16 20:05:22.656533 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:22.656498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv" event={"ID":"6604e787-06da-439a-ab8a-50bf84278da6","Type":"ContainerStarted","Data":"901fd64436497a287f60cbde83f7acf1864597b083e471c3ca5976519fe8ebf5"}
Jul 16 20:05:23.290286 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.290229    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n2bdv" podStartSLOduration=2.290209812 podStartE2EDuration="2.290209812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:05:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:05:22.67839764 +0000 UTC m=+286.826493046" watchObservedRunningTime="2026-07-16 20:05:23.290209812 +0000 UTC m=+287.438305234"
Jul 16 20:05:23.292020 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.291991    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tddh7"]
Jul 16 20:05:23.294525 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.294510    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tddh7"
Jul 16 20:05:23.297375 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.297354    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ktsj9\""
Jul 16 20:05:23.317190 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.317165    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tddh7"]
Jul 16 20:05:23.397878 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.397847    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8fe7a27-ca1b-46a0-8acd-68186b9fc507-bound-sa-token\") pod \"cert-manager-55b4d954d5-tddh7\" (UID: \"a8fe7a27-ca1b-46a0-8acd-68186b9fc507\") " pod="cert-manager/cert-manager-55b4d954d5-tddh7"
Jul 16 20:05:23.398025 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.397903    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqnrp\" (UniqueName: \"kubernetes.io/projected/a8fe7a27-ca1b-46a0-8acd-68186b9fc507-kube-api-access-nqnrp\") pod \"cert-manager-55b4d954d5-tddh7\" (UID: \"a8fe7a27-ca1b-46a0-8acd-68186b9fc507\") " pod="cert-manager/cert-manager-55b4d954d5-tddh7"
Jul 16 20:05:23.499473 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.499421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8fe7a27-ca1b-46a0-8acd-68186b9fc507-bound-sa-token\") pod \"cert-manager-55b4d954d5-tddh7\" (UID: \"a8fe7a27-ca1b-46a0-8acd-68186b9fc507\") " pod="cert-manager/cert-manager-55b4d954d5-tddh7"
Jul 16 20:05:23.499617 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.499488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqnrp\" (UniqueName: \"kubernetes.io/projected/a8fe7a27-ca1b-46a0-8acd-68186b9fc507-kube-api-access-nqnrp\") pod \"cert-manager-55b4d954d5-tddh7\" (UID: \"a8fe7a27-ca1b-46a0-8acd-68186b9fc507\") " pod="cert-manager/cert-manager-55b4d954d5-tddh7"
Jul 16 20:05:23.507647 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.507624    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8fe7a27-ca1b-46a0-8acd-68186b9fc507-bound-sa-token\") pod \"cert-manager-55b4d954d5-tddh7\" (UID: \"a8fe7a27-ca1b-46a0-8acd-68186b9fc507\") " pod="cert-manager/cert-manager-55b4d954d5-tddh7"
Jul 16 20:05:23.507743 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.507723    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqnrp\" (UniqueName: \"kubernetes.io/projected/a8fe7a27-ca1b-46a0-8acd-68186b9fc507-kube-api-access-nqnrp\") pod \"cert-manager-55b4d954d5-tddh7\" (UID: \"a8fe7a27-ca1b-46a0-8acd-68186b9fc507\") " pod="cert-manager/cert-manager-55b4d954d5-tddh7"
Jul 16 20:05:23.603776 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.603709    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tddh7"
Jul 16 20:05:23.638510 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.638476    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9dwjt"
Jul 16 20:05:23.733080 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:23.733056    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tddh7"]
Jul 16 20:05:23.735552 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:05:23.735526    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8fe7a27_ca1b_46a0_8acd_68186b9fc507.slice/crio-a8758cdf1b97ba81c3406262953d308c66f465c0ea46f0be797bf39f4244df0f WatchSource:0}: Error finding container a8758cdf1b97ba81c3406262953d308c66f465c0ea46f0be797bf39f4244df0f: Status 404 returned error can't find the container with id a8758cdf1b97ba81c3406262953d308c66f465c0ea46f0be797bf39f4244df0f
Jul 16 20:05:24.674256 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:24.674217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tddh7" event={"ID":"a8fe7a27-ca1b-46a0-8acd-68186b9fc507","Type":"ContainerStarted","Data":"8051d7c7424e9049c58600642b47918ad59dbb711dbfce714cd994f818e84d68"}
Jul 16 20:05:24.674256 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:24.674261    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tddh7" event={"ID":"a8fe7a27-ca1b-46a0-8acd-68186b9fc507","Type":"ContainerStarted","Data":"a8758cdf1b97ba81c3406262953d308c66f465c0ea46f0be797bf39f4244df0f"}
Jul 16 20:05:24.690211 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:24.690167    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-tddh7" podStartSLOduration=1.6901494750000001 podStartE2EDuration="1.690149475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:05:24.688390912 +0000 UTC m=+288.836486334" watchObservedRunningTime="2026-07-16 20:05:24.690149475 +0000 UTC m=+288.838244899"
Jul 16 20:05:26.041343 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.041309    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"]
Jul 16 20:05:26.043626 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.043610    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"
Jul 16 20:05:26.045681 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.045649    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:05:26.045775 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.045743    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lrfnv\""
Jul 16 20:05:26.046478 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.046460    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 16 20:05:26.051857 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.051835    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"]
Jul 16 20:05:26.118709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.118680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d9sp\" (UniqueName: \"kubernetes.io/projected/8d851ff9-3382-45dc-b3d0-9b0c1c28d88e-kube-api-access-5d9sp\") pod \"openshift-lws-operator-fd8ccff4c-5289b\" (UID: \"8d851ff9-3382-45dc-b3d0-9b0c1c28d88e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"
Jul 16 20:05:26.118788 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.118720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d851ff9-3382-45dc-b3d0-9b0c1c28d88e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5289b\" (UID: \"8d851ff9-3382-45dc-b3d0-9b0c1c28d88e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"
Jul 16 20:05:26.219792 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.219766    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d9sp\" (UniqueName: \"kubernetes.io/projected/8d851ff9-3382-45dc-b3d0-9b0c1c28d88e-kube-api-access-5d9sp\") pod \"openshift-lws-operator-fd8ccff4c-5289b\" (UID: \"8d851ff9-3382-45dc-b3d0-9b0c1c28d88e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"
Jul 16 20:05:26.219897 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.219801    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d851ff9-3382-45dc-b3d0-9b0c1c28d88e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5289b\" (UID: \"8d851ff9-3382-45dc-b3d0-9b0c1c28d88e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"
Jul 16 20:05:26.220133 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.220118    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d851ff9-3382-45dc-b3d0-9b0c1c28d88e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5289b\" (UID: \"8d851ff9-3382-45dc-b3d0-9b0c1c28d88e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"
Jul 16 20:05:26.227824 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.227798    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d9sp\" (UniqueName: \"kubernetes.io/projected/8d851ff9-3382-45dc-b3d0-9b0c1c28d88e-kube-api-access-5d9sp\") pod \"openshift-lws-operator-fd8ccff4c-5289b\" (UID: \"8d851ff9-3382-45dc-b3d0-9b0c1c28d88e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"
Jul 16 20:05:26.352958 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.352893    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"
Jul 16 20:05:26.482283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.482257    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b"]
Jul 16 20:05:26.484782 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:05:26.484748    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d851ff9_3382_45dc_b3d0_9b0c1c28d88e.slice/crio-cf9b0bff2f3f16efbfc4a0c6d675d05ece4506f070dede1142ba760e7c466c1d WatchSource:0}: Error finding container cf9b0bff2f3f16efbfc4a0c6d675d05ece4506f070dede1142ba760e7c466c1d: Status 404 returned error can't find the container with id cf9b0bff2f3f16efbfc4a0c6d675d05ece4506f070dede1142ba760e7c466c1d
Jul 16 20:05:26.684302 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:26.684268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b" event={"ID":"8d851ff9-3382-45dc-b3d0-9b0c1c28d88e","Type":"ContainerStarted","Data":"cf9b0bff2f3f16efbfc4a0c6d675d05ece4506f070dede1142ba760e7c466c1d"}
Jul 16 20:05:29.701469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:29.701411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b" event={"ID":"8d851ff9-3382-45dc-b3d0-9b0c1c28d88e","Type":"ContainerStarted","Data":"81ab90665303cc5637f41ced0b51a96df96b92b7cfdaf0a6a95131b7d63d827e"}
Jul 16 20:05:29.718057 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:29.718006    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5289b" podStartSLOduration=1.175379506 podStartE2EDuration="3.717991231s" totalImagesPullingTime="2.542611725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:05:29.716732919 +0000 UTC m=+293.864828343" watchObservedRunningTime="2026-07-16 20:05:29.717991231 +0000 UTC m=+293.866086654"
Jul 16 20:05:33.896627 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:33.896593    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"]
Jul 16 20:05:33.899403 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:33.899379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:33.901619 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:33.901594    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 16 20:05:33.901718 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:33.901642    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g8prd\""
Jul 16 20:05:33.901792 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:33.901710    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 16 20:05:33.907582 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:33.907557    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"]
Jul 16 20:05:33.979559 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:33.979534    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4kft\" (UniqueName: \"kubernetes.io/projected/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-kube-api-access-l4kft\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:33.979705 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:33.979572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:33.979705 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:33.979599    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:34.080861 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.080831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4kft\" (UniqueName: \"kubernetes.io/projected/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-kube-api-access-l4kft\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:34.080983 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.080870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:34.080983 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.080903    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:34.081225 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.081207    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:34.081269 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.081252    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:34.090442 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.090411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4kft\" (UniqueName: \"kubernetes.io/projected/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-kube-api-access-l4kft\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:34.210319 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.210295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:34.333920 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.333831    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"]
Jul 16 20:05:34.336335 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:05:34.336309    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5d5f028_c2d7_44b8_9ed3_efebf02ca70c.slice/crio-c2155feb4f460d9d339bbec51e424685e7233a0179762684aba809f48f8a3e38 WatchSource:0}: Error finding container c2155feb4f460d9d339bbec51e424685e7233a0179762684aba809f48f8a3e38: Status 404 returned error can't find the container with id c2155feb4f460d9d339bbec51e424685e7233a0179762684aba809f48f8a3e38
Jul 16 20:05:34.725069 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.725030    2579 generic.go:356] "Generic (PLEG): container finished" podID="f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" containerID="8cee8040f5696c56300b8d734e1cbe9624089c29c5d5249218e2c6578ca48ffd" exitCode=0
Jul 16 20:05:34.725261 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.725088    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd" event={"ID":"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c","Type":"ContainerDied","Data":"8cee8040f5696c56300b8d734e1cbe9624089c29c5d5249218e2c6578ca48ffd"}
Jul 16 20:05:34.725261 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:34.725138    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd" event={"ID":"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c","Type":"ContainerStarted","Data":"c2155feb4f460d9d339bbec51e424685e7233a0179762684aba809f48f8a3e38"}
Jul 16 20:05:36.238128 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:36.238100    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 20:05:36.734912 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:36.734870    2579 generic.go:356] "Generic (PLEG): container finished" podID="f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" containerID="4847a7a2eef5315576b49d1a648c6175ae089eceddd9fd3541bdfc3775fbf3d0" exitCode=0
Jul 16 20:05:36.736093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:36.734935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd" event={"ID":"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c","Type":"ContainerDied","Data":"4847a7a2eef5315576b49d1a648c6175ae089eceddd9fd3541bdfc3775fbf3d0"}
Jul 16 20:05:37.741402 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:37.741360    2579 generic.go:356] "Generic (PLEG): container finished" podID="f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" containerID="5d20d31e657b1dcfb1853666f08d8528b8842bad4e91f3dc53554bb1bb68a8ce" exitCode=0
Jul 16 20:05:37.741858 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:37.741416    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd" event={"ID":"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c","Type":"ContainerDied","Data":"5d20d31e657b1dcfb1853666f08d8528b8842bad4e91f3dc53554bb1bb68a8ce"}
Jul 16 20:05:38.868831 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:38.868806    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:38.919636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:38.919607    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4kft\" (UniqueName: \"kubernetes.io/projected/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-kube-api-access-l4kft\") pod \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") "
Jul 16 20:05:38.919797 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:38.919686    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-bundle\") pod \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") "
Jul 16 20:05:38.919797 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:38.919731    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-util\") pod \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\" (UID: \"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c\") "
Jul 16 20:05:38.920427 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:38.920400    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-bundle" (OuterVolumeSpecName: "bundle") pod "f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" (UID: "f5d5f028-c2d7-44b8-9ed3-efebf02ca70c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:05:38.921839 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:38.921814    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-kube-api-access-l4kft" (OuterVolumeSpecName: "kube-api-access-l4kft") pod "f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" (UID: "f5d5f028-c2d7-44b8-9ed3-efebf02ca70c"). InnerVolumeSpecName "kube-api-access-l4kft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:05:38.925176 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:38.925156    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-util" (OuterVolumeSpecName: "util") pod "f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" (UID: "f5d5f028-c2d7-44b8-9ed3-efebf02ca70c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:05:39.020249 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:39.020174    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4kft\" (UniqueName: \"kubernetes.io/projected/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-kube-api-access-l4kft\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:05:39.020249 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:39.020203    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:05:39.020249 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:39.020216    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5d5f028-c2d7-44b8-9ed3-efebf02ca70c-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:05:39.751691 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:39.751662    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd"
Jul 16 20:05:39.751847 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:39.751692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affxlqd" event={"ID":"f5d5f028-c2d7-44b8-9ed3-efebf02ca70c","Type":"ContainerDied","Data":"c2155feb4f460d9d339bbec51e424685e7233a0179762684aba809f48f8a3e38"}
Jul 16 20:05:39.751847 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:39.751733    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2155feb4f460d9d339bbec51e424685e7233a0179762684aba809f48f8a3e38"
Jul 16 20:05:42.612153 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.612118    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"]
Jul 16 20:05:42.612636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.612600    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" containerName="pull"
Jul 16 20:05:42.612636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.612620    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" containerName="pull"
Jul 16 20:05:42.612755 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.612642    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" containerName="extract"
Jul 16 20:05:42.612755 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.612652    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" containerName="extract"
Jul 16 20:05:42.612755 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.612671    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" containerName="util"
Jul 16 20:05:42.612755 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.612680    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" containerName="util"
Jul 16 20:05:42.612943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.612791    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5d5f028-c2d7-44b8-9ed3-efebf02ca70c" containerName="extract"
Jul 16 20:05:42.614833 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.614812    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.618696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.618673    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 16 20:05:42.618696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.618687    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 16 20:05:42.618856 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.618703    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-sl5q7\""
Jul 16 20:05:42.618856 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.618675    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 16 20:05:42.625648 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.625627    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"]
Jul 16 20:05:42.648804 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.648771    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m45q9\" (UniqueName: \"kubernetes.io/projected/20e16a21-db23-4613-aba4-27c1f832a509-kube-api-access-m45q9\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.648933 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.648808    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/20e16a21-db23-4613-aba4-27c1f832a509-manager-config\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.648933 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.648831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/20e16a21-db23-4613-aba4-27c1f832a509-metrics-cert\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.648933 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.648889    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20e16a21-db23-4613-aba4-27c1f832a509-cert\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.750132 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.750094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/20e16a21-db23-4613-aba4-27c1f832a509-metrics-cert\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.750132 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.750134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20e16a21-db23-4613-aba4-27c1f832a509-cert\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.750387 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.750181    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m45q9\" (UniqueName: \"kubernetes.io/projected/20e16a21-db23-4613-aba4-27c1f832a509-kube-api-access-m45q9\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.750387 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.750206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/20e16a21-db23-4613-aba4-27c1f832a509-manager-config\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.751009 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.750989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/20e16a21-db23-4613-aba4-27c1f832a509-manager-config\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.752781 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.752751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/20e16a21-db23-4613-aba4-27c1f832a509-metrics-cert\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.752871 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.752803    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20e16a21-db23-4613-aba4-27c1f832a509-cert\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.760904 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.760885    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m45q9\" (UniqueName: \"kubernetes.io/projected/20e16a21-db23-4613-aba4-27c1f832a509-kube-api-access-m45q9\") pod \"lws-controller-manager-7b8db6765-dds9b\" (UID: \"20e16a21-db23-4613-aba4-27c1f832a509\") " pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:42.924775 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:42.924743    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:43.067163 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:43.067142    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"]
Jul 16 20:05:43.068795 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:05:43.068771    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20e16a21_db23_4613_aba4_27c1f832a509.slice/crio-423c81f8285400ad244da5798473cd8ea8994eab79d1b96b8b84bcfd8a044318 WatchSource:0}: Error finding container 423c81f8285400ad244da5798473cd8ea8994eab79d1b96b8b84bcfd8a044318: Status 404 returned error can't find the container with id 423c81f8285400ad244da5798473cd8ea8994eab79d1b96b8b84bcfd8a044318
Jul 16 20:05:43.070938 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:43.070920    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 20:05:43.771405 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:43.771368    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b" event={"ID":"20e16a21-db23-4613-aba4-27c1f832a509","Type":"ContainerStarted","Data":"423c81f8285400ad244da5798473cd8ea8994eab79d1b96b8b84bcfd8a044318"}
Jul 16 20:05:45.508088 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.508051    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"]
Jul 16 20:05:45.510336 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.510319    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.516011 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:05:45.515982    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="dd439004-1886-4e2e-bdac-ee5c110a0a07" pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.516011 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.515993    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 20:05:45.516574 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.516553    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 20:05:45.516974 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.516952    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p5l6r\""
Jul 16 20:05:45.517256 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.517238    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 20:05:45.518829 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.518810    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 20:05:45.551714 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.551691    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"]
Jul 16 20:05:45.574088 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.574065    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvkpw\" (UniqueName: \"kubernetes.io/projected/dd439004-1886-4e2e-bdac-ee5c110a0a07-kube-api-access-lvkpw\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.574188 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.574101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd439004-1886-4e2e-bdac-ee5c110a0a07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.574188 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.574126    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dd439004-1886-4e2e-bdac-ee5c110a0a07-manifests\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.574188 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.574145    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd439004-1886-4e2e-bdac-ee5c110a0a07-tmp\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.574289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.574212    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd439004-1886-4e2e-bdac-ee5c110a0a07-webhook-cert\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.675615 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.675575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvkpw\" (UniqueName: \"kubernetes.io/projected/dd439004-1886-4e2e-bdac-ee5c110a0a07-kube-api-access-lvkpw\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.675756 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.675625    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd439004-1886-4e2e-bdac-ee5c110a0a07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.675756 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.675650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dd439004-1886-4e2e-bdac-ee5c110a0a07-manifests\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.675756 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.675679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd439004-1886-4e2e-bdac-ee5c110a0a07-tmp\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.675756 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.675703    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd439004-1886-4e2e-bdac-ee5c110a0a07-webhook-cert\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.676103 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.676074    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dd439004-1886-4e2e-bdac-ee5c110a0a07-manifests\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.678103 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.678081    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd439004-1886-4e2e-bdac-ee5c110a0a07-tmp\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.678326 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.678308    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd439004-1886-4e2e-bdac-ee5c110a0a07-webhook-cert\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.678384 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.678307    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd439004-1886-4e2e-bdac-ee5c110a0a07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.687446 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.687416    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvkpw\" (UniqueName: \"kubernetes.io/projected/dd439004-1886-4e2e-bdac-ee5c110a0a07-kube-api-access-lvkpw\") pod \"opendatahub-operator-controller-manager-64c594d8d4-9vdc5\" (UID: \"dd439004-1886-4e2e-bdac-ee5c110a0a07\") " pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.781789 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.781725    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b" event={"ID":"20e16a21-db23-4613-aba4-27c1f832a509","Type":"ContainerStarted","Data":"575161f4c10e96ae35450333e6b120b244b7f45527b928e788ca6957004d90d0"}
Jul 16 20:05:45.781875 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.781849    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:05:45.811087 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.811031    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b" podStartSLOduration=1.920785185 podStartE2EDuration="3.811017207s" totalImagesPullingTime="1.890232022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:05:45.808784987 +0000 UTC m=+309.956880411" watchObservedRunningTime="2026-07-16 20:05:45.811017207 +0000 UTC m=+309.959112630"
Jul 16 20:05:45.820110 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.820082    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:45.951789 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:45.951754    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"]
Jul 16 20:05:45.956255 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:05:45.956217    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd439004_1886_4e2e_bdac_ee5c110a0a07.slice/crio-3eaf3d250bd81ee02c7c2ea9093ff3a07172321741065d03a57c5e54e8e32fa8 WatchSource:0}: Error finding container 3eaf3d250bd81ee02c7c2ea9093ff3a07172321741065d03a57c5e54e8e32fa8: Status 404 returned error can't find the container with id 3eaf3d250bd81ee02c7c2ea9093ff3a07172321741065d03a57c5e54e8e32fa8
Jul 16 20:05:46.787789 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:46.787752    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5" event={"ID":"dd439004-1886-4e2e-bdac-ee5c110a0a07","Type":"ContainerStarted","Data":"3eaf3d250bd81ee02c7c2ea9093ff3a07172321741065d03a57c5e54e8e32fa8"}
Jul 16 20:05:51.814217 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:51.814136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5" event={"ID":"dd439004-1886-4e2e-bdac-ee5c110a0a07","Type":"ContainerStarted","Data":"62ec63cd68e2efeb1a783ea0f74e2295c0af314d3623731acc726cdaa7d01f79"}
Jul 16 20:05:52.822553 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:52.822516    2579 generic.go:356] "Generic (PLEG): container finished" podID="dd439004-1886-4e2e-bdac-ee5c110a0a07" containerID="62ec63cd68e2efeb1a783ea0f74e2295c0af314d3623731acc726cdaa7d01f79" exitCode=0
Jul 16 20:05:52.823051 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:52.822570    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5" event={"ID":"dd439004-1886-4e2e-bdac-ee5c110a0a07","Type":"ContainerDied","Data":"62ec63cd68e2efeb1a783ea0f74e2295c0af314d3623731acc726cdaa7d01f79"}
Jul 16 20:05:53.829067 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:53.829025    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5" event={"ID":"dd439004-1886-4e2e-bdac-ee5c110a0a07","Type":"ContainerStarted","Data":"8a31be848257865ec651559bab923bb34ddbcff9bcfa3e78623a007f181a4e11"}
Jul 16 20:05:53.829476 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:53.829142    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:05:53.851544 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:53.851495    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5" podStartSLOduration=1.826601275 podStartE2EDuration="8.851481373s" totalImagesPullingTime="5.976429051s" totalInitContainerRuntime="1.048451047s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:05:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:05:53.848861434 +0000 UTC m=+317.996956858" watchObservedRunningTime="2026-07-16 20:05:53.851481373 +0000 UTC m=+317.999576793"
Jul 16 20:05:56.790147 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:05:56.790115    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b8db6765-dds9b"
Jul 16 20:06:04.836211 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:04.836177    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64c594d8d4-9vdc5"
Jul 16 20:06:15.179477 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.179425    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"]
Jul 16 20:06:15.183781 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.183758    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.186187 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.186163    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 16 20:06:15.188055 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.187355    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 20:06:15.188055 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.187697    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 20:06:15.188055 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.187774    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7pgj9\""
Jul 16 20:06:15.188055 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.187898    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 16 20:06:15.188055 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.187966    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 16 20:06:15.188416 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.188321    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 16 20:06:15.189014 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.188993    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"]
Jul 16 20:06:15.339191 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.339153    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.339191 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.339195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.339477 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.339274    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.339477 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.339308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.339477 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.339392    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x54t\" (UniqueName: \"kubernetes.io/projected/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-kube-api-access-7x54t\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.339477 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.339421    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.339477 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.339467    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.440659 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.440560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.440659 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.440597    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.440659 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.440623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x54t\" (UniqueName: \"kubernetes.io/projected/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-kube-api-access-7x54t\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.440946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.440741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.440946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.440791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.440946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.440872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.440946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.440923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.441224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.441205    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.443090 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.443059    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.443207 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.443091    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.443207 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.443171    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.443207 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.443196    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.456574 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.456553    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x54t\" (UniqueName: \"kubernetes.io/projected/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-kube-api-access-7x54t\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.456695 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.456591    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2eea5a7-b748-4e9a-91a2-10a6b82d87b9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7rz45\" (UID: \"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.503514 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.503484    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:15.658982 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.658957    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"]
Jul 16 20:06:15.660882 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:06:15.660856    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2eea5a7_b748_4e9a_91a2_10a6b82d87b9.slice/crio-65adee947042e63df129af0bf0c83bdc199297fdb151a40bf246b0b2d339ef99 WatchSource:0}: Error finding container 65adee947042e63df129af0bf0c83bdc199297fdb151a40bf246b0b2d339ef99: Status 404 returned error can't find the container with id 65adee947042e63df129af0bf0c83bdc199297fdb151a40bf246b0b2d339ef99
Jul 16 20:06:15.934403 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:15.934367    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45" event={"ID":"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9","Type":"ContainerStarted","Data":"65adee947042e63df129af0bf0c83bdc199297fdb151a40bf246b0b2d339ef99"}
Jul 16 20:06:20.615773 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:20.615732    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:06:20.616056 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:20.615799    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:06:20.960540 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:20.960497    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45" event={"ID":"c2eea5a7-b748-4e9a-91a2-10a6b82d87b9","Type":"ContainerStarted","Data":"e96a827ce377e5af50d2cf022a14f7cd5a75b4a3fe323d4395f0c56a9ae24730"}
Jul 16 20:06:20.960725 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:20.960570    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:20.983862 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:20.983805    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45" podStartSLOduration=1.031048657 podStartE2EDuration="5.983786544s" totalImagesPullingTime="4.952737887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:06:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:06:20.982901781 +0000 UTC m=+345.130997205" watchObservedRunningTime="2026-07-16 20:06:20.983786544 +0000 UTC m=+345.131881967"
Jul 16 20:06:21.968175 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:21.968145    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7rz45"
Jul 16 20:06:59.839249 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:59.839218    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b786d75f6-27729"]
Jul 16 20:06:59.841512 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:59.841490    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:06:59.855379 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:59.855342    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b786d75f6-27729"]
Jul 16 20:06:59.899969 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:59.899938    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-console-config\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:06:59.900098 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:59.899982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-console-oauth-config\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:06:59.900098 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:59.900010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-trusted-ca-bundle\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:06:59.900098 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:59.900069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2z7n\" (UniqueName: \"kubernetes.io/projected/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-kube-api-access-r2z7n\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:06:59.900098 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:59.900096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-service-ca\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:06:59.900269 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:59.900135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-console-serving-cert\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:06:59.900269 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:06:59.900192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-oauth-serving-cert\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.000760 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.000730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2z7n\" (UniqueName: \"kubernetes.io/projected/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-kube-api-access-r2z7n\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.000894 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.000769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-service-ca\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.000894 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.000795    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-console-serving-cert\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.000894 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.000815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-oauth-serving-cert\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.000894 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.000854    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-console-config\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.000894 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.000880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-console-oauth-config\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.001168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.000899    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-trusted-ca-bundle\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.001607 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.001577    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-oauth-serving-cert\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.001719 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.001606    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-service-ca\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.001719 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.001684    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-console-config\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.001793 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.001763    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-trusted-ca-bundle\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.003204 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.003183    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-console-serving-cert\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.003354 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.003332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-console-oauth-config\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.012119 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.012095    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2z7n\" (UniqueName: \"kubernetes.io/projected/2f84a6f0-8a18-4f41-8586-27cf56f70ec0-kube-api-access-r2z7n\") pod \"console-7b786d75f6-27729\" (UID: \"2f84a6f0-8a18-4f41-8586-27cf56f70ec0\") " pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.150584 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.150485    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:00.280449 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:00.280312    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b786d75f6-27729"]
Jul 16 20:07:00.283067 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:07:00.283038    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f84a6f0_8a18_4f41_8586_27cf56f70ec0.slice/crio-8d35eb8bbbca348018cb97265507c25a3d052b1dc01b1e8c533bd11642bcc6b4 WatchSource:0}: Error finding container 8d35eb8bbbca348018cb97265507c25a3d052b1dc01b1e8c533bd11642bcc6b4: Status 404 returned error can't find the container with id 8d35eb8bbbca348018cb97265507c25a3d052b1dc01b1e8c533bd11642bcc6b4
Jul 16 20:07:01.146877 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:01.146842    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b786d75f6-27729" event={"ID":"2f84a6f0-8a18-4f41-8586-27cf56f70ec0","Type":"ContainerStarted","Data":"00e989cbe8b171222cb64500ca99d0b3c516f535d3f00c14c2b8a8c96c73cdd9"}
Jul 16 20:07:01.147233 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:01.146891    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b786d75f6-27729" event={"ID":"2f84a6f0-8a18-4f41-8586-27cf56f70ec0","Type":"ContainerStarted","Data":"8d35eb8bbbca348018cb97265507c25a3d052b1dc01b1e8c533bd11642bcc6b4"}
Jul 16 20:07:01.167529 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:01.167475    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b786d75f6-27729" podStartSLOduration=2.167457078 podStartE2EDuration="2.167457078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:06:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:07:01.165897758 +0000 UTC m=+385.313993184" watchObservedRunningTime="2026-07-16 20:07:01.167457078 +0000 UTC m=+385.315552503"
Jul 16 20:07:10.150619 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:10.150587    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:10.151031 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:10.150634    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:10.155216 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:10.155193    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:10.195418 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:10.195393    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b786d75f6-27729"
Jul 16 20:07:10.255593 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:10.255565    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77c4cb7b55-wn9d2"]
Jul 16 20:07:24.306419 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.306384    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"]
Jul 16 20:07:24.309552 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.309531    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.311956 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.311933    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 20:07:24.312095 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.311972    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-slrcx\""
Jul 16 20:07:24.312563 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.312545    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 20:07:24.322746 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.322719    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"]
Jul 16 20:07:24.406688 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.406654    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.406866 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.406781    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj9rg\" (UniqueName: \"kubernetes.io/projected/48e55e72-0725-434d-bd51-c5bb9a0e64a2-kube-api-access-pj9rg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.406866 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.406816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.507637 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.507592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj9rg\" (UniqueName: \"kubernetes.io/projected/48e55e72-0725-434d-bd51-c5bb9a0e64a2-kube-api-access-pj9rg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.507637 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.507636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.507866 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.507666    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.508002 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.507986    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.508051 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.508022    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.515588 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.515566    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj9rg\" (UniqueName: \"kubernetes.io/projected/48e55e72-0725-434d-bd51-c5bb9a0e64a2-kube-api-access-pj9rg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.619840 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.619758    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:24.716510 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.716394    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"]
Jul 16 20:07:24.720383 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.720363    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:24.728372 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.728349    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"]
Jul 16 20:07:24.748972 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.748953    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"]
Jul 16 20:07:24.751510 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:07:24.751486    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e55e72_0725_434d_bd51_c5bb9a0e64a2.slice/crio-ed1dbdf247c94f89910625647fc4e82785fdeeadffc093928908d8e4efa3265d WatchSource:0}: Error finding container ed1dbdf247c94f89910625647fc4e82785fdeeadffc093928908d8e4efa3265d: Status 404 returned error can't find the container with id ed1dbdf247c94f89910625647fc4e82785fdeeadffc093928908d8e4efa3265d
Jul 16 20:07:24.810058 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.810038    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:24.810138 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.810072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n9lp\" (UniqueName: \"kubernetes.io/projected/ee7a12b4-7f78-47c6-a283-9a706cdad756-kube-api-access-7n9lp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:24.810138 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.810122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:24.910583 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.910516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n9lp\" (UniqueName: \"kubernetes.io/projected/ee7a12b4-7f78-47c6-a283-9a706cdad756-kube-api-access-7n9lp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:24.910583 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.910582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:24.910717 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.910638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:24.910939 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.910920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:24.910979 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.910949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:24.919110 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:24.919087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n9lp\" (UniqueName: \"kubernetes.io/projected/ee7a12b4-7f78-47c6-a283-9a706cdad756-kube-api-access-7n9lp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:25.032068 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.032041    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:25.161289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.161260    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"]
Jul 16 20:07:25.163614 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:07:25.163587    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7a12b4_7f78_47c6_a283_9a706cdad756.slice/crio-aa3cbd09c5662725c301b129a7c24643566359f71c10ef3090dc4cd966eb11fa WatchSource:0}: Error finding container aa3cbd09c5662725c301b129a7c24643566359f71c10ef3090dc4cd966eb11fa: Status 404 returned error can't find the container with id aa3cbd09c5662725c301b129a7c24643566359f71c10ef3090dc4cd966eb11fa
Jul 16 20:07:25.262365 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.262335    2579 generic.go:356] "Generic (PLEG): container finished" podID="48e55e72-0725-434d-bd51-c5bb9a0e64a2" containerID="efbcea52523d72ec64d63ac5f5d60469cb1dd48576264073d435f9130b949b85" exitCode=0
Jul 16 20:07:25.262514 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.262418    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45" event={"ID":"48e55e72-0725-434d-bd51-c5bb9a0e64a2","Type":"ContainerDied","Data":"efbcea52523d72ec64d63ac5f5d60469cb1dd48576264073d435f9130b949b85"}
Jul 16 20:07:25.262514 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.262500    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45" event={"ID":"48e55e72-0725-434d-bd51-c5bb9a0e64a2","Type":"ContainerStarted","Data":"ed1dbdf247c94f89910625647fc4e82785fdeeadffc093928908d8e4efa3265d"}
Jul 16 20:07:25.264003 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.263980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs" event={"ID":"ee7a12b4-7f78-47c6-a283-9a706cdad756","Type":"ContainerStarted","Data":"e898bbb1d3983c508a81631c33845624dddd597ec0b68cdc39813869b43301a3"}
Jul 16 20:07:25.264093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.264023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs" event={"ID":"ee7a12b4-7f78-47c6-a283-9a706cdad756","Type":"ContainerStarted","Data":"aa3cbd09c5662725c301b129a7c24643566359f71c10ef3090dc4cd966eb11fa"}
Jul 16 20:07:25.311508 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.311483    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"]
Jul 16 20:07:25.314773 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.314753    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.323575 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.323552    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"]
Jul 16 20:07:25.414928 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.414900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjl5t\" (UniqueName: \"kubernetes.io/projected/b448c7f8-6112-4834-adad-30534db365e9-kube-api-access-qjl5t\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.415090 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.414935    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.415090 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.414969    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.515761 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.515730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjl5t\" (UniqueName: \"kubernetes.io/projected/b448c7f8-6112-4834-adad-30534db365e9-kube-api-access-qjl5t\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.515887 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.515776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.515887 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.515825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.516165 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.516148    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.516246 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.516228    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.524903 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.524875    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjl5t\" (UniqueName: \"kubernetes.io/projected/b448c7f8-6112-4834-adad-30534db365e9-kube-api-access-qjl5t\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.624090 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.624065    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:25.754832 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.754805    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"]
Jul 16 20:07:25.756611 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:07:25.756581    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb448c7f8_6112_4834_adad_30534db365e9.slice/crio-08d000cfd6058395e4067fb6672b2809eb2d4721adddb1c3916a1ee2b0c5b297 WatchSource:0}: Error finding container 08d000cfd6058395e4067fb6672b2809eb2d4721adddb1c3916a1ee2b0c5b297: Status 404 returned error can't find the container with id 08d000cfd6058395e4067fb6672b2809eb2d4721adddb1c3916a1ee2b0c5b297
Jul 16 20:07:25.913101 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.913072    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"]
Jul 16 20:07:25.916806 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.916791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:25.918230 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.918211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4f22\" (UniqueName: \"kubernetes.io/projected/1758a2d6-6453-47f2-b2aa-f947a054bf68-kube-api-access-t4f22\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:25.918290 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.918277    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:25.918332 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.918321    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:25.929112 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:25.929088    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"]
Jul 16 20:07:26.018704 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.018664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:26.018864 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.018723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4f22\" (UniqueName: \"kubernetes.io/projected/1758a2d6-6453-47f2-b2aa-f947a054bf68-kube-api-access-t4f22\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:26.018864 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.018772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:26.019107 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.019086    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:26.019178 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.019115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:26.027368 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.027342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4f22\" (UniqueName: \"kubernetes.io/projected/1758a2d6-6453-47f2-b2aa-f947a054bf68-kube-api-access-t4f22\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:26.225474 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.225390    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:26.275682 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.275653    2579 generic.go:356] "Generic (PLEG): container finished" podID="ee7a12b4-7f78-47c6-a283-9a706cdad756" containerID="e898bbb1d3983c508a81631c33845624dddd597ec0b68cdc39813869b43301a3" exitCode=0
Jul 16 20:07:26.275782 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.275719    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs" event={"ID":"ee7a12b4-7f78-47c6-a283-9a706cdad756","Type":"ContainerDied","Data":"e898bbb1d3983c508a81631c33845624dddd597ec0b68cdc39813869b43301a3"}
Jul 16 20:07:26.277082 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.277060    2579 generic.go:356] "Generic (PLEG): container finished" podID="b448c7f8-6112-4834-adad-30534db365e9" containerID="2467b4bc6421c997dde59f0e295a1eebd1373a5e57e43b20fe47e703490eefca" exitCode=0
Jul 16 20:07:26.277173 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.277143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx" event={"ID":"b448c7f8-6112-4834-adad-30534db365e9","Type":"ContainerDied","Data":"2467b4bc6421c997dde59f0e295a1eebd1373a5e57e43b20fe47e703490eefca"}
Jul 16 20:07:26.277237 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.277194    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx" event={"ID":"b448c7f8-6112-4834-adad-30534db365e9","Type":"ContainerStarted","Data":"08d000cfd6058395e4067fb6672b2809eb2d4721adddb1c3916a1ee2b0c5b297"}
Jul 16 20:07:26.381970 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:26.381937    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"]
Jul 16 20:07:26.465191 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:07:26.465161    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1758a2d6_6453_47f2_b2aa_f947a054bf68.slice/crio-85c852a4afa423862e251afe605dd11c0943e64532eb849e4b7bc5e65f9d82b4 WatchSource:0}: Error finding container 85c852a4afa423862e251afe605dd11c0943e64532eb849e4b7bc5e65f9d82b4: Status 404 returned error can't find the container with id 85c852a4afa423862e251afe605dd11c0943e64532eb849e4b7bc5e65f9d82b4
Jul 16 20:07:27.283653 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:27.283617    2579 generic.go:356] "Generic (PLEG): container finished" podID="1758a2d6-6453-47f2-b2aa-f947a054bf68" containerID="8fffd9ae259b076c6742c807c5547915e1f59bc4e313ab2af74cbc726524b869" exitCode=0
Jul 16 20:07:27.283799 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:27.283699    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw" event={"ID":"1758a2d6-6453-47f2-b2aa-f947a054bf68","Type":"ContainerDied","Data":"8fffd9ae259b076c6742c807c5547915e1f59bc4e313ab2af74cbc726524b869"}
Jul 16 20:07:27.283799 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:27.283754    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw" event={"ID":"1758a2d6-6453-47f2-b2aa-f947a054bf68","Type":"ContainerStarted","Data":"85c852a4afa423862e251afe605dd11c0943e64532eb849e4b7bc5e65f9d82b4"}
Jul 16 20:07:27.285554 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:27.285534    2579 generic.go:356] "Generic (PLEG): container finished" podID="48e55e72-0725-434d-bd51-c5bb9a0e64a2" containerID="17d112d679ca6c55af78733993dcdb1f602141a73941b1cb02a601149a2aa62e" exitCode=0
Jul 16 20:07:27.285649 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:27.285624    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45" event={"ID":"48e55e72-0725-434d-bd51-c5bb9a0e64a2","Type":"ContainerDied","Data":"17d112d679ca6c55af78733993dcdb1f602141a73941b1cb02a601149a2aa62e"}
Jul 16 20:07:27.287698 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:27.287677    2579 generic.go:356] "Generic (PLEG): container finished" podID="ee7a12b4-7f78-47c6-a283-9a706cdad756" containerID="20fff46ce03deaaf7cb71f5a2e2f464b297dfaad92af71bd398f6b4bb9ed962a" exitCode=0
Jul 16 20:07:27.287804 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:27.287712    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs" event={"ID":"ee7a12b4-7f78-47c6-a283-9a706cdad756","Type":"ContainerDied","Data":"20fff46ce03deaaf7cb71f5a2e2f464b297dfaad92af71bd398f6b4bb9ed962a"}
Jul 16 20:07:27.289558 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:27.289536    2579 generic.go:356] "Generic (PLEG): container finished" podID="b448c7f8-6112-4834-adad-30534db365e9" containerID="33406a50f55f6858928c677352831673dc29cc1312f035b01b79e8090b05d886" exitCode=0
Jul 16 20:07:27.289641 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:27.289597    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx" event={"ID":"b448c7f8-6112-4834-adad-30534db365e9","Type":"ContainerDied","Data":"33406a50f55f6858928c677352831673dc29cc1312f035b01b79e8090b05d886"}
Jul 16 20:07:28.296363 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:28.296290    2579 generic.go:356] "Generic (PLEG): container finished" podID="1758a2d6-6453-47f2-b2aa-f947a054bf68" containerID="70dcb2730ab045e3cb20eff6dad491228ffcef833a00a795280025a660996afe" exitCode=0
Jul 16 20:07:28.296781 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:28.296355    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw" event={"ID":"1758a2d6-6453-47f2-b2aa-f947a054bf68","Type":"ContainerDied","Data":"70dcb2730ab045e3cb20eff6dad491228ffcef833a00a795280025a660996afe"}
Jul 16 20:07:28.298534 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:28.298511    2579 generic.go:356] "Generic (PLEG): container finished" podID="48e55e72-0725-434d-bd51-c5bb9a0e64a2" containerID="9f0a6207a6ea159295a236f398e5fb12b706c729d200cb19f4ffe7519ea18e95" exitCode=0
Jul 16 20:07:28.298632 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:28.298548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45" event={"ID":"48e55e72-0725-434d-bd51-c5bb9a0e64a2","Type":"ContainerDied","Data":"9f0a6207a6ea159295a236f398e5fb12b706c729d200cb19f4ffe7519ea18e95"}
Jul 16 20:07:28.300616 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:28.300598    2579 generic.go:356] "Generic (PLEG): container finished" podID="ee7a12b4-7f78-47c6-a283-9a706cdad756" containerID="5514b2968aeeacf11e3d134169671da3d482304788dc1295046740273e3464ad" exitCode=0
Jul 16 20:07:28.300689 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:28.300662    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs" event={"ID":"ee7a12b4-7f78-47c6-a283-9a706cdad756","Type":"ContainerDied","Data":"5514b2968aeeacf11e3d134169671da3d482304788dc1295046740273e3464ad"}
Jul 16 20:07:28.302719 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:28.302698    2579 generic.go:356] "Generic (PLEG): container finished" podID="b448c7f8-6112-4834-adad-30534db365e9" containerID="8bb42d49624278fd466ae14a1d3d3ca42700f384629036e013cac014657f92dc" exitCode=0
Jul 16 20:07:28.302788 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:28.302760    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx" event={"ID":"b448c7f8-6112-4834-adad-30534db365e9","Type":"ContainerDied","Data":"8bb42d49624278fd466ae14a1d3d3ca42700f384629036e013cac014657f92dc"}
Jul 16 20:07:29.309879 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.309841    2579 generic.go:356] "Generic (PLEG): container finished" podID="1758a2d6-6453-47f2-b2aa-f947a054bf68" containerID="91101fe2da9c8330eb39fe0ddc224092a4721dafa84f05545a1e4276a7e7980b" exitCode=0
Jul 16 20:07:29.310330 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.309926    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw" event={"ID":"1758a2d6-6453-47f2-b2aa-f947a054bf68","Type":"ContainerDied","Data":"91101fe2da9c8330eb39fe0ddc224092a4721dafa84f05545a1e4276a7e7980b"}
Jul 16 20:07:29.445837 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.445810    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:29.504562 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.504541    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:29.507463 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.507427    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:29.545499 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.545477    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-util\") pod \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") "
Jul 16 20:07:29.545601 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.545521    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-util\") pod \"ee7a12b4-7f78-47c6-a283-9a706cdad756\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") "
Jul 16 20:07:29.545601 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.545545    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-bundle\") pod \"ee7a12b4-7f78-47c6-a283-9a706cdad756\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") "
Jul 16 20:07:29.545601 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.545569    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-util\") pod \"b448c7f8-6112-4834-adad-30534db365e9\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") "
Jul 16 20:07:29.545758 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.545612    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-bundle\") pod \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") "
Jul 16 20:07:29.545758 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.545639    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7n9lp\" (UniqueName: \"kubernetes.io/projected/ee7a12b4-7f78-47c6-a283-9a706cdad756-kube-api-access-7n9lp\") pod \"ee7a12b4-7f78-47c6-a283-9a706cdad756\" (UID: \"ee7a12b4-7f78-47c6-a283-9a706cdad756\") "
Jul 16 20:07:29.545758 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.545675    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-bundle\") pod \"b448c7f8-6112-4834-adad-30534db365e9\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") "
Jul 16 20:07:29.545758 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.545722    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjl5t\" (UniqueName: \"kubernetes.io/projected/b448c7f8-6112-4834-adad-30534db365e9-kube-api-access-qjl5t\") pod \"b448c7f8-6112-4834-adad-30534db365e9\" (UID: \"b448c7f8-6112-4834-adad-30534db365e9\") "
Jul 16 20:07:29.545758 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.545754    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj9rg\" (UniqueName: \"kubernetes.io/projected/48e55e72-0725-434d-bd51-c5bb9a0e64a2-kube-api-access-pj9rg\") pod \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\" (UID: \"48e55e72-0725-434d-bd51-c5bb9a0e64a2\") "
Jul 16 20:07:29.546724 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.546077    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-bundle" (OuterVolumeSpecName: "bundle") pod "48e55e72-0725-434d-bd51-c5bb9a0e64a2" (UID: "48e55e72-0725-434d-bd51-c5bb9a0e64a2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:07:29.546724 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.546402    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-bundle" (OuterVolumeSpecName: "bundle") pod "ee7a12b4-7f78-47c6-a283-9a706cdad756" (UID: "ee7a12b4-7f78-47c6-a283-9a706cdad756"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:07:29.546724 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.546676    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-bundle" (OuterVolumeSpecName: "bundle") pod "b448c7f8-6112-4834-adad-30534db365e9" (UID: "b448c7f8-6112-4834-adad-30534db365e9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:07:29.548744 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.548717    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee7a12b4-7f78-47c6-a283-9a706cdad756-kube-api-access-7n9lp" (OuterVolumeSpecName: "kube-api-access-7n9lp") pod "ee7a12b4-7f78-47c6-a283-9a706cdad756" (UID: "ee7a12b4-7f78-47c6-a283-9a706cdad756"). InnerVolumeSpecName "kube-api-access-7n9lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:07:29.548848 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.548755    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b448c7f8-6112-4834-adad-30534db365e9-kube-api-access-qjl5t" (OuterVolumeSpecName: "kube-api-access-qjl5t") pod "b448c7f8-6112-4834-adad-30534db365e9" (UID: "b448c7f8-6112-4834-adad-30534db365e9"). InnerVolumeSpecName "kube-api-access-qjl5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:07:29.548913 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.548847    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e55e72-0725-434d-bd51-c5bb9a0e64a2-kube-api-access-pj9rg" (OuterVolumeSpecName: "kube-api-access-pj9rg") pod "48e55e72-0725-434d-bd51-c5bb9a0e64a2" (UID: "48e55e72-0725-434d-bd51-c5bb9a0e64a2"). InnerVolumeSpecName "kube-api-access-pj9rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:07:29.552496 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.552474    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-util" (OuterVolumeSpecName: "util") pod "ee7a12b4-7f78-47c6-a283-9a706cdad756" (UID: "ee7a12b4-7f78-47c6-a283-9a706cdad756"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:07:29.553656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.553615    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-util" (OuterVolumeSpecName: "util") pod "b448c7f8-6112-4834-adad-30534db365e9" (UID: "b448c7f8-6112-4834-adad-30534db365e9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:07:29.554030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.554005    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-util" (OuterVolumeSpecName: "util") pod "48e55e72-0725-434d-bd51-c5bb9a0e64a2" (UID: "48e55e72-0725-434d-bd51-c5bb9a0e64a2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:07:29.647059 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.646979    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:29.647059 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.647007    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:29.647059 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.647020    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n9lp\" (UniqueName: \"kubernetes.io/projected/ee7a12b4-7f78-47c6-a283-9a706cdad756-kube-api-access-7n9lp\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:29.647059 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.647032    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b448c7f8-6112-4834-adad-30534db365e9-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:29.647059 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.647043    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjl5t\" (UniqueName: \"kubernetes.io/projected/b448c7f8-6112-4834-adad-30534db365e9-kube-api-access-qjl5t\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:29.647059 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.647055    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj9rg\" (UniqueName: \"kubernetes.io/projected/48e55e72-0725-434d-bd51-c5bb9a0e64a2-kube-api-access-pj9rg\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:29.647314 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.647066    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48e55e72-0725-434d-bd51-c5bb9a0e64a2-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:29.647314 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.647078    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:29.647314 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:29.647087    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee7a12b4-7f78-47c6-a283-9a706cdad756-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:30.316739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.316698    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs" event={"ID":"ee7a12b4-7f78-47c6-a283-9a706cdad756","Type":"ContainerDied","Data":"aa3cbd09c5662725c301b129a7c24643566359f71c10ef3090dc4cd966eb11fa"}
Jul 16 20:07:30.317162 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.316746    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa3cbd09c5662725c301b129a7c24643566359f71c10ef3090dc4cd966eb11fa"
Jul 16 20:07:30.317162 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.316708    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs"
Jul 16 20:07:30.318505 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.318475    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx"
Jul 16 20:07:30.318615 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.318505    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx" event={"ID":"b448c7f8-6112-4834-adad-30534db365e9","Type":"ContainerDied","Data":"08d000cfd6058395e4067fb6672b2809eb2d4721adddb1c3916a1ee2b0c5b297"}
Jul 16 20:07:30.318615 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.318548    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08d000cfd6058395e4067fb6672b2809eb2d4721adddb1c3916a1ee2b0c5b297"
Jul 16 20:07:30.320254 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.320239    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45"
Jul 16 20:07:30.320359 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.320266    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45" event={"ID":"48e55e72-0725-434d-bd51-c5bb9a0e64a2","Type":"ContainerDied","Data":"ed1dbdf247c94f89910625647fc4e82785fdeeadffc093928908d8e4efa3265d"}
Jul 16 20:07:30.320359 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.320302    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed1dbdf247c94f89910625647fc4e82785fdeeadffc093928908d8e4efa3265d"
Jul 16 20:07:30.448056 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.448033    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:30.454416 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.454395    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4f22\" (UniqueName: \"kubernetes.io/projected/1758a2d6-6453-47f2-b2aa-f947a054bf68-kube-api-access-t4f22\") pod \"1758a2d6-6453-47f2-b2aa-f947a054bf68\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") "
Jul 16 20:07:30.454528 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.454482    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-bundle\") pod \"1758a2d6-6453-47f2-b2aa-f947a054bf68\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") "
Jul 16 20:07:30.454528 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.454507    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-util\") pod \"1758a2d6-6453-47f2-b2aa-f947a054bf68\" (UID: \"1758a2d6-6453-47f2-b2aa-f947a054bf68\") "
Jul 16 20:07:30.454979 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.454951    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-bundle" (OuterVolumeSpecName: "bundle") pod "1758a2d6-6453-47f2-b2aa-f947a054bf68" (UID: "1758a2d6-6453-47f2-b2aa-f947a054bf68"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:07:30.456494 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.456472    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1758a2d6-6453-47f2-b2aa-f947a054bf68-kube-api-access-t4f22" (OuterVolumeSpecName: "kube-api-access-t4f22") pod "1758a2d6-6453-47f2-b2aa-f947a054bf68" (UID: "1758a2d6-6453-47f2-b2aa-f947a054bf68"). InnerVolumeSpecName "kube-api-access-t4f22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:07:30.459725 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.459702    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-util" (OuterVolumeSpecName: "util") pod "1758a2d6-6453-47f2-b2aa-f947a054bf68" (UID: "1758a2d6-6453-47f2-b2aa-f947a054bf68"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:07:30.556125 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.556077    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4f22\" (UniqueName: \"kubernetes.io/projected/1758a2d6-6453-47f2-b2aa-f947a054bf68-kube-api-access-t4f22\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:30.556125 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.556123    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:30.556125 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:30.556140    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1758a2d6-6453-47f2-b2aa-f947a054bf68-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:31.327034 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:31.327006    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw"
Jul 16 20:07:31.327034 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:31.327012    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw" event={"ID":"1758a2d6-6453-47f2-b2aa-f947a054bf68","Type":"ContainerDied","Data":"85c852a4afa423862e251afe605dd11c0943e64532eb849e4b7bc5e65f9d82b4"}
Jul 16 20:07:31.327552 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:31.327059    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85c852a4afa423862e251afe605dd11c0943e64532eb849e4b7bc5e65f9d82b4"
Jul 16 20:07:35.282989 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.282940    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77c4cb7b55-wn9d2" podUID="25dc3a5f-fecc-4e19-b1ad-d373849eebef" containerName="console" containerID="cri-o://1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e" gracePeriod=15
Jul 16 20:07:35.522958 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.522936    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:07:35.598554 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.598487    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-trusted-ca-bundle\") pod \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") "
Jul 16 20:07:35.598554 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.598534    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-oauth-config\") pod \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") "
Jul 16 20:07:35.598740 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.598570    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-oauth-serving-cert\") pod \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") "
Jul 16 20:07:35.598740 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.598600    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2ft8\" (UniqueName: \"kubernetes.io/projected/25dc3a5f-fecc-4e19-b1ad-d373849eebef-kube-api-access-l2ft8\") pod \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") "
Jul 16 20:07:35.598740 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.598645    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-service-ca\") pod \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") "
Jul 16 20:07:35.598740 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.598682    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-serving-cert\") pod \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") "
Jul 16 20:07:35.598938 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.598755    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-config\") pod \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\" (UID: \"25dc3a5f-fecc-4e19-b1ad-d373849eebef\") "
Jul 16 20:07:35.599017 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.598991    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "25dc3a5f-fecc-4e19-b1ad-d373849eebef" (UID: "25dc3a5f-fecc-4e19-b1ad-d373849eebef"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:07:35.599121 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.599093    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "25dc3a5f-fecc-4e19-b1ad-d373849eebef" (UID: "25dc3a5f-fecc-4e19-b1ad-d373849eebef"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:07:35.599239 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.599180    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-service-ca" (OuterVolumeSpecName: "service-ca") pod "25dc3a5f-fecc-4e19-b1ad-d373849eebef" (UID: "25dc3a5f-fecc-4e19-b1ad-d373849eebef"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:07:35.599239 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.599224    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-config" (OuterVolumeSpecName: "console-config") pod "25dc3a5f-fecc-4e19-b1ad-d373849eebef" (UID: "25dc3a5f-fecc-4e19-b1ad-d373849eebef"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:07:35.600765 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.600744    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "25dc3a5f-fecc-4e19-b1ad-d373849eebef" (UID: "25dc3a5f-fecc-4e19-b1ad-d373849eebef"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:07:35.601038 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.601013    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25dc3a5f-fecc-4e19-b1ad-d373849eebef-kube-api-access-l2ft8" (OuterVolumeSpecName: "kube-api-access-l2ft8") pod "25dc3a5f-fecc-4e19-b1ad-d373849eebef" (UID: "25dc3a5f-fecc-4e19-b1ad-d373849eebef"). InnerVolumeSpecName "kube-api-access-l2ft8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:07:35.601038 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.601021    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "25dc3a5f-fecc-4e19-b1ad-d373849eebef" (UID: "25dc3a5f-fecc-4e19-b1ad-d373849eebef"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:07:35.699664 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.699636    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-serving-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:35.699664 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.699662    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-config\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:35.699810 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.699671    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-trusted-ca-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:35.699810 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.699680    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25dc3a5f-fecc-4e19-b1ad-d373849eebef-console-oauth-config\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:35.699810 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.699690    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-oauth-serving-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:35.699810 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.699698    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2ft8\" (UniqueName: \"kubernetes.io/projected/25dc3a5f-fecc-4e19-b1ad-d373849eebef-kube-api-access-l2ft8\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:35.699810 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:35.699707    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25dc3a5f-fecc-4e19-b1ad-d373849eebef-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:07:36.351798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:36.351763    2579 generic.go:356] "Generic (PLEG): container finished" podID="25dc3a5f-fecc-4e19-b1ad-d373849eebef" containerID="1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e" exitCode=0
Jul 16 20:07:36.352171 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:36.351802    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c4cb7b55-wn9d2"
Jul 16 20:07:36.352171 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:36.351835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c4cb7b55-wn9d2" event={"ID":"25dc3a5f-fecc-4e19-b1ad-d373849eebef","Type":"ContainerDied","Data":"1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e"}
Jul 16 20:07:36.352171 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:36.351882    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c4cb7b55-wn9d2" event={"ID":"25dc3a5f-fecc-4e19-b1ad-d373849eebef","Type":"ContainerDied","Data":"e54a8a4f2d76037fa1e8cb6a6ec0f7868cf729d530e71f81e3667a9b3d8d61d3"}
Jul 16 20:07:36.352171 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:36.351893    2579 scope.go:117] "RemoveContainer" containerID="1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e"
Jul 16 20:07:36.360629 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:36.360612    2579 scope.go:117] "RemoveContainer" containerID="1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e"
Jul 16 20:07:36.360876 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:07:36.360857    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e\": container with ID starting with 1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e not found: ID does not exist" containerID="1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e"
Jul 16 20:07:36.360970 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:36.360882    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e"} err="failed to get container status \"1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e\": rpc error: code = NotFound desc = could not find container \"1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e\": container with ID starting with 1542152fb5736fb12ab14b3b2cbe0d5e62d1872a7ded16a71cb3b08fbcac636e not found: ID does not exist"
Jul 16 20:07:36.372614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:36.372592    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77c4cb7b55-wn9d2"]
Jul 16 20:07:36.376128 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:36.376108    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77c4cb7b55-wn9d2"]
Jul 16 20:07:36.404486 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:07:36.404457    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25dc3a5f-fecc-4e19-b1ad-d373849eebef" path="/var/lib/kubelet/pods/25dc3a5f-fecc-4e19-b1ad-d373849eebef/volumes"
Jul 16 20:08:58.723514 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723473    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723825    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee7a12b4-7f78-47c6-a283-9a706cdad756" containerName="util"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723836    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee7a12b4-7f78-47c6-a283-9a706cdad756" containerName="util"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723845    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48e55e72-0725-434d-bd51-c5bb9a0e64a2" containerName="extract"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723851    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="48e55e72-0725-434d-bd51-c5bb9a0e64a2" containerName="extract"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723860    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1758a2d6-6453-47f2-b2aa-f947a054bf68" containerName="pull"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723866    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1758a2d6-6453-47f2-b2aa-f947a054bf68" containerName="pull"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723876    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b448c7f8-6112-4834-adad-30534db365e9" containerName="util"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723881    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b448c7f8-6112-4834-adad-30534db365e9" containerName="util"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723888    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee7a12b4-7f78-47c6-a283-9a706cdad756" containerName="pull"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723893    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee7a12b4-7f78-47c6-a283-9a706cdad756" containerName="pull"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723898    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee7a12b4-7f78-47c6-a283-9a706cdad756" containerName="extract"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723904    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee7a12b4-7f78-47c6-a283-9a706cdad756" containerName="extract"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723911    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b448c7f8-6112-4834-adad-30534db365e9" containerName="extract"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723916    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b448c7f8-6112-4834-adad-30534db365e9" containerName="extract"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723923    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1758a2d6-6453-47f2-b2aa-f947a054bf68" containerName="extract"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723929    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1758a2d6-6453-47f2-b2aa-f947a054bf68" containerName="extract"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723936    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25dc3a5f-fecc-4e19-b1ad-d373849eebef" containerName="console"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723941    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="25dc3a5f-fecc-4e19-b1ad-d373849eebef" containerName="console"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723947    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48e55e72-0725-434d-bd51-c5bb9a0e64a2" containerName="util"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723953    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="48e55e72-0725-434d-bd51-c5bb9a0e64a2" containerName="util"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723959    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48e55e72-0725-434d-bd51-c5bb9a0e64a2" containerName="pull"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723965    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="48e55e72-0725-434d-bd51-c5bb9a0e64a2" containerName="pull"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723972    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1758a2d6-6453-47f2-b2aa-f947a054bf68" containerName="util"
Jul 16 20:08:58.723968 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723977    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1758a2d6-6453-47f2-b2aa-f947a054bf68" containerName="util"
Jul 16 20:08:58.724703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723988    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b448c7f8-6112-4834-adad-30534db365e9" containerName="pull"
Jul 16 20:08:58.724703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.723993    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b448c7f8-6112-4834-adad-30534db365e9" containerName="pull"
Jul 16 20:08:58.724703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.724050    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b448c7f8-6112-4834-adad-30534db365e9" containerName="extract"
Jul 16 20:08:58.724703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.724059    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="25dc3a5f-fecc-4e19-b1ad-d373849eebef" containerName="console"
Jul 16 20:08:58.724703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.724066    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1758a2d6-6453-47f2-b2aa-f947a054bf68" containerName="extract"
Jul 16 20:08:58.724703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.724073    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="48e55e72-0725-434d-bd51-c5bb9a0e64a2" containerName="extract"
Jul 16 20:08:58.724703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.724079    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee7a12b4-7f78-47c6-a283-9a706cdad756" containerName="extract"
Jul 16 20:08:58.727122 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.727104    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h"
Jul 16 20:08:58.735255 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.735233    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-slrcx\""
Jul 16 20:08:58.735719 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.735703    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 20:08:58.736095 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.736075    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 20:08:58.736195 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.736175    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 16 20:08:58.749834 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.749813    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:08:58.770978 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.770951    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:08:58.844681 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.844648    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/04363917-f191-4630-85b3-3532f8019d61-config-file\") pod \"limitador-limitador-54d6d988d5-9x54h\" (UID: \"04363917-f191-4630-85b3-3532f8019d61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h"
Jul 16 20:08:58.844827 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.844768    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz8tw\" (UniqueName: \"kubernetes.io/projected/04363917-f191-4630-85b3-3532f8019d61-kube-api-access-kz8tw\") pod \"limitador-limitador-54d6d988d5-9x54h\" (UID: \"04363917-f191-4630-85b3-3532f8019d61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h"
Jul 16 20:08:58.945795 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.945758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz8tw\" (UniqueName: \"kubernetes.io/projected/04363917-f191-4630-85b3-3532f8019d61-kube-api-access-kz8tw\") pod \"limitador-limitador-54d6d988d5-9x54h\" (UID: \"04363917-f191-4630-85b3-3532f8019d61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h"
Jul 16 20:08:58.945977 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.945820    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/04363917-f191-4630-85b3-3532f8019d61-config-file\") pod \"limitador-limitador-54d6d988d5-9x54h\" (UID: \"04363917-f191-4630-85b3-3532f8019d61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h"
Jul 16 20:08:58.946482 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.946426    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/04363917-f191-4630-85b3-3532f8019d61-config-file\") pod \"limitador-limitador-54d6d988d5-9x54h\" (UID: \"04363917-f191-4630-85b3-3532f8019d61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h"
Jul 16 20:08:58.956539 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:58.956512    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz8tw\" (UniqueName: \"kubernetes.io/projected/04363917-f191-4630-85b3-3532f8019d61-kube-api-access-kz8tw\") pod \"limitador-limitador-54d6d988d5-9x54h\" (UID: \"04363917-f191-4630-85b3-3532f8019d61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h"
Jul 16 20:08:59.037084 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:59.037015    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h"
Jul 16 20:08:59.207395 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:59.207286    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:08:59.210108 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:08:59.210079    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04363917_f191_4630_85b3_3532f8019d61.slice/crio-4b8c68f4e516d9de6c5f7a289a00dbd637ff1dd33707c5686672bc0931e7525b WatchSource:0}: Error finding container 4b8c68f4e516d9de6c5f7a289a00dbd637ff1dd33707c5686672bc0931e7525b: Status 404 returned error can't find the container with id 4b8c68f4e516d9de6c5f7a289a00dbd637ff1dd33707c5686672bc0931e7525b
Jul 16 20:08:59.782498 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:08:59.782460    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h" event={"ID":"04363917-f191-4630-85b3-3532f8019d61","Type":"ContainerStarted","Data":"4b8c68f4e516d9de6c5f7a289a00dbd637ff1dd33707c5686672bc0931e7525b"}
Jul 16 20:09:02.801281 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:09:02.801239    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h" event={"ID":"04363917-f191-4630-85b3-3532f8019d61","Type":"ContainerStarted","Data":"87ff646591060a1c266783997610016bb7c4ac78e68845c01b72f827c2f1ad24"}
Jul 16 20:09:02.801695 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:09:02.801310    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h"
Jul 16 20:09:02.822502 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:09:02.822455    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h" podStartSLOduration=2.179041822 podStartE2EDuration="4.822416018s" totalImagesPullingTime="2.643374196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:09:02.821518824 +0000 UTC m=+506.969614250" watchObservedRunningTime="2026-07-16 20:09:02.822416018 +0000 UTC m=+506.970511441"
Jul 16 20:09:13.807620 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:09:13.807589    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-9x54h"
Jul 16 20:10:18.893646 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:18.893616    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-nsfxk"]
Jul 16 20:10:18.896943 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:18.896926    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk"
Jul 16 20:10:18.899186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:18.899166    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lvsf2\""
Jul 16 20:10:18.899299 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:18.899236    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 16 20:10:18.909600 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:18.909580    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-nsfxk"]
Jul 16 20:10:18.940964 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:18.940931    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:10:19.030629 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:19.030604    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a192e8a0-baa2-4783-a097-bb862e2859a2-plugins-config-volume\") pod \"payload-processing-5bf47cd988-nsfxk\" (UID: \"a192e8a0-baa2-4783-a097-bb862e2859a2\") " pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk"
Jul 16 20:10:19.030738 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:19.030657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44q68\" (UniqueName: \"kubernetes.io/projected/a192e8a0-baa2-4783-a097-bb862e2859a2-kube-api-access-44q68\") pod \"payload-processing-5bf47cd988-nsfxk\" (UID: \"a192e8a0-baa2-4783-a097-bb862e2859a2\") " pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk"
Jul 16 20:10:19.131186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:19.131158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a192e8a0-baa2-4783-a097-bb862e2859a2-plugins-config-volume\") pod \"payload-processing-5bf47cd988-nsfxk\" (UID: \"a192e8a0-baa2-4783-a097-bb862e2859a2\") " pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk"
Jul 16 20:10:19.131350 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:19.131211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44q68\" (UniqueName: \"kubernetes.io/projected/a192e8a0-baa2-4783-a097-bb862e2859a2-kube-api-access-44q68\") pod \"payload-processing-5bf47cd988-nsfxk\" (UID: \"a192e8a0-baa2-4783-a097-bb862e2859a2\") " pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk"
Jul 16 20:10:19.131904 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:19.131880    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a192e8a0-baa2-4783-a097-bb862e2859a2-plugins-config-volume\") pod \"payload-processing-5bf47cd988-nsfxk\" (UID: \"a192e8a0-baa2-4783-a097-bb862e2859a2\") " pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk"
Jul 16 20:10:19.139764 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:19.139743    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44q68\" (UniqueName: \"kubernetes.io/projected/a192e8a0-baa2-4783-a097-bb862e2859a2-kube-api-access-44q68\") pod \"payload-processing-5bf47cd988-nsfxk\" (UID: \"a192e8a0-baa2-4783-a097-bb862e2859a2\") " pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk"
Jul 16 20:10:19.207959 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:19.207903    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk"
Jul 16 20:10:19.348478 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:19.348445    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-nsfxk"]
Jul 16 20:10:19.349946 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:10:19.349917    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda192e8a0_baa2_4783_a097_bb862e2859a2.slice/crio-a41811950c1672bbef6b1365a0daff66575fc5cf0ffdfb5276f11ab931334e8d WatchSource:0}: Error finding container a41811950c1672bbef6b1365a0daff66575fc5cf0ffdfb5276f11ab931334e8d: Status 404 returned error can't find the container with id a41811950c1672bbef6b1365a0daff66575fc5cf0ffdfb5276f11ab931334e8d
Jul 16 20:10:20.201137 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:20.201097    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk" event={"ID":"a192e8a0-baa2-4783-a097-bb862e2859a2","Type":"ContainerStarted","Data":"a41811950c1672bbef6b1365a0daff66575fc5cf0ffdfb5276f11ab931334e8d"}
Jul 16 20:10:21.212331 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:21.212296    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk" event={"ID":"a192e8a0-baa2-4783-a097-bb862e2859a2","Type":"ContainerStarted","Data":"2acc653db944d077d6e7aea90ecd61a490fb276af0382311374fb44f72bfffc8"}
Jul 16 20:10:21.212728 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:21.212451    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk"
Jul 16 20:10:21.228590 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:21.227948    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk" podStartSLOduration=1.6965760749999999 podStartE2EDuration="3.227931029s" totalImagesPullingTime="1.531354954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:10:21.227204174 +0000 UTC m=+585.375299597" watchObservedRunningTime="2026-07-16 20:10:21.227931029 +0000 UTC m=+585.376026453"
Jul 16 20:10:32.220144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:32.220108    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-nsfxk"
Jul 16 20:10:47.691950 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:47.691916    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:10:53.378736 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:10:53.378695    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:11:07.220951 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.220912    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"]
Jul 16 20:11:07.224618 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.224599    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.226960 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.226938    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 20:11:07.227740 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.227723    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5xhqz\""
Jul 16 20:11:07.227886 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.227744    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 20:11:07.227886 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.227744    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 16 20:11:07.235680 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.235655    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"]
Jul 16 20:11:07.352656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.352617    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.352865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.352659    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbnvs\" (UniqueName: \"kubernetes.io/projected/9b79b743-e531-47dc-a537-7bd99093365b-kube-api-access-jbnvs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.352865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.352695    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.352865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.352772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b79b743-e531-47dc-a537-7bd99093365b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.352865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.352796    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.353034 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.352871    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.454116 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.454079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.454116 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.454119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbnvs\" (UniqueName: \"kubernetes.io/projected/9b79b743-e531-47dc-a537-7bd99093365b-kube-api-access-jbnvs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.454382 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.454147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.454382 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.454204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b79b743-e531-47dc-a537-7bd99093365b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.454382 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.454228    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.454382 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.454280    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.454626 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.454556    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.454626 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.454591    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.454721 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.454645    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.456389 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.456372    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b79b743-e531-47dc-a537-7bd99093365b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.456673 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.456654    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b79b743-e531-47dc-a537-7bd99093365b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.461395 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.461371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbnvs\" (UniqueName: \"kubernetes.io/projected/9b79b743-e531-47dc-a537-7bd99093365b-kube-api-access-jbnvs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9v4f9\" (UID: \"9b79b743-e531-47dc-a537-7bd99093365b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.536092 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.536000    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:07.671370 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.671345    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"]
Jul 16 20:11:07.673952 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:11:07.673925    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b79b743_e531_47dc_a537_7bd99093365b.slice/crio-3d414d07e6acbb0d3f38bd3fb3d78309f11e6eecd9504f700eb76d067be7ac71 WatchSource:0}: Error finding container 3d414d07e6acbb0d3f38bd3fb3d78309f11e6eecd9504f700eb76d067be7ac71: Status 404 returned error can't find the container with id 3d414d07e6acbb0d3f38bd3fb3d78309f11e6eecd9504f700eb76d067be7ac71
Jul 16 20:11:07.675226 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:07.675206    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 20:11:08.456005 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:08.455960    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9" event={"ID":"9b79b743-e531-47dc-a537-7bd99093365b","Type":"ContainerStarted","Data":"3d414d07e6acbb0d3f38bd3fb3d78309f11e6eecd9504f700eb76d067be7ac71"}
Jul 16 20:11:08.487393 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:08.487351    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:11:09.463077 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:09.463045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9" event={"ID":"9b79b743-e531-47dc-a537-7bd99093365b","Type":"ContainerStarted","Data":"9ce44e3349267ee0ef8dc6b87c3eac041087e556b5872c867a85c13cffbda505"}
Jul 16 20:11:09.480986 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:09.480933    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9" podStartSLOduration=1.440510407 podStartE2EDuration="2.480914175s" totalImagesPullingTime="1.040403768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:11:09.479398001 +0000 UTC m=+633.627493441" watchObservedRunningTime="2026-07-16 20:11:09.480914175 +0000 UTC m=+633.629009599"
Jul 16 20:11:17.536546 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:17.536506    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:17.536969 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:17.536623    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:17.548962 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:17.548924    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:18.519495 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:18.519466    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9v4f9"
Jul 16 20:11:20.286199 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:20.286164    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:11:33.195613 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:33.195513    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:11:42.978782 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:11:42.978740    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:12:52.085129 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:12:52.085090    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:13:01.691282 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:13:01.691243    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:13:12.680273 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:13:12.680241    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:14:54.240160 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:14:54.240112    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:15:35.679838 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:15:35.679799    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:15:43.184424 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:15:43.184389    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:15:54.280347 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:15:54.280309    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:16:02.889496 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:16:02.889460    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:16:13.493937 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:16:13.493900    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:16:24.787866 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:16:24.787827    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:16:35.789687 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:16:35.789645    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:16:45.497688 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:16:45.497647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:17:11.080018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:17:11.079979    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:17:23.789128 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:17:23.789088    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:17:47.097196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:17:47.097156    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:17:57.991026 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:17:57.990991    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:18:14.786593 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:18:14.786558    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:18:24.594302 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:18:24.594270    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:18:35.204164 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:18:35.204074    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:18:46.181444 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:18:46.181402    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:18:59.686091 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:18:59.686058    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:20:52.784452 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:20:52.784394    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:22:41.596841 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:22:41.596800    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:22:54.001343 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:22:54.001307    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:23:03.202001 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:23:03.201969    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:23:16.185569 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:23:16.185533    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:23:26.182073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:23:26.182036    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:23:36.489447 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:23:36.489394    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:23:44.581674 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:23:44.581646    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:23:55.984069 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:23:55.984028    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:24:06.682995 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:24:06.682960    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:24:15.579932 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:24:15.579878    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:24:27.892978 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:24:27.892941    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:24:50.883487 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:24:50.883456    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:24:58.387213 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:24:58.387175    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:25:08.484157 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:25:08.484125    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:25:18.599425 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:25:18.599392    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:25:27.478190 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:25:27.478159    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:25:39.873517 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:25:39.873481    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:25:54.290669 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:25:54.290636    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:26:03.789444 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:26:03.789406    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:26:16.410354 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:26:16.410314    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:26:29.016634 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:26:29.016596    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:26:47.104450 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:26:47.104398    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:27:04.691023 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:27:04.690987    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:27:14.587706 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:27:14.587670    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:27:26.076382 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:27:26.076346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:27:36.792264 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:27:36.792189    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:27:48.997322 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:27:48.997287    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:28:01.700765 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:01.700728    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:28:11.378948 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:11.378912    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:28:18.787675 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:18.787633    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:28:32.157586 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:32.157550    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:28:48.586091 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:48.586048    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:28:52.954077 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:52.954044    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"]
Jul 16 20:28:52.957385 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:52.957368    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:52.959701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:52.959680    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-49aaf8ce-serving-cert\""
Jul 16 20:28:52.959828 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:52.959745    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 20:28:52.960481 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:52.960462    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g78x2\""
Jul 16 20:28:52.960589 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:52.960488    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 20:28:52.969329 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:52.969304    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"]
Jul 16 20:28:53.004958 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.004926    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c485e0-89bc-4c09-8a4e-c63823a79d5d-openshift-service-ca\") pod \"maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:53.004958 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.004963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c485e0-89bc-4c09-8a4e-c63823a79d5d-maas-api-tls\") pod \"maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:53.005149 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.005021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcqqm\" (UniqueName: \"kubernetes.io/projected/92c485e0-89bc-4c09-8a4e-c63823a79d5d-kube-api-access-lcqqm\") pod \"maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:53.106073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.106034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c485e0-89bc-4c09-8a4e-c63823a79d5d-openshift-service-ca\") pod \"maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:53.106073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.106074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c485e0-89bc-4c09-8a4e-c63823a79d5d-maas-api-tls\") pod \"maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:53.106291 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.106113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcqqm\" (UniqueName: \"kubernetes.io/projected/92c485e0-89bc-4c09-8a4e-c63823a79d5d-kube-api-access-lcqqm\") pod \"maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:53.106291 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:28:53.106238    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-serving-cert: secret "maas-api-e2e-ait-49aaf8ce-serving-cert" not found
Jul 16 20:28:53.106364 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:28:53.106314    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c485e0-89bc-4c09-8a4e-c63823a79d5d-maas-api-tls podName:92c485e0-89bc-4c09-8a4e-c63823a79d5d nodeName:}" failed. No retries permitted until 2026-07-16 20:28:53.606298677 +0000 UTC m=+1697.754394077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/92c485e0-89bc-4c09-8a4e-c63823a79d5d-maas-api-tls") pod "maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb" (UID: "92c485e0-89bc-4c09-8a4e-c63823a79d5d") : secret "maas-api-e2e-ait-49aaf8ce-serving-cert" not found
Jul 16 20:28:53.106688 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.106668    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c485e0-89bc-4c09-8a4e-c63823a79d5d-openshift-service-ca\") pod \"maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:53.115469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.115426    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcqqm\" (UniqueName: \"kubernetes.io/projected/92c485e0-89bc-4c09-8a4e-c63823a79d5d-kube-api-access-lcqqm\") pod \"maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:53.610740 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.610701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c485e0-89bc-4c09-8a4e-c63823a79d5d-maas-api-tls\") pod \"maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:53.613180 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.613154    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c485e0-89bc-4c09-8a4e-c63823a79d5d-maas-api-tls\") pod \"maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:53.868149 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:53.868049    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:54.000489 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:54.000458    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"]
Jul 16 20:28:54.002900 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:28:54.002870    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c485e0_89bc_4c09_8a4e_c63823a79d5d.slice/crio-f2ec744277ac096a33809c33aea5e1a0c02a43debc9e23d34624a772a67f24f7 WatchSource:0}: Error finding container f2ec744277ac096a33809c33aea5e1a0c02a43debc9e23d34624a772a67f24f7: Status 404 returned error can't find the container with id f2ec744277ac096a33809c33aea5e1a0c02a43debc9e23d34624a772a67f24f7
Jul 16 20:28:54.004585 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:54.004565    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 20:28:54.936103 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:54.936065    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb" event={"ID":"92c485e0-89bc-4c09-8a4e-c63823a79d5d","Type":"ContainerStarted","Data":"f2ec744277ac096a33809c33aea5e1a0c02a43debc9e23d34624a772a67f24f7"}
Jul 16 20:28:57.678926 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.676092    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd"]
Jul 16 20:28:57.681228 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.681200    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd"
Jul 16 20:28:57.683736 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.683711    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:28:57.685533 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.685509    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd"]
Jul 16 20:28:57.748853 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.748817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/772ee840-6fdb-48be-9648-3f6ac86aa33b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd\" (UID: \"772ee840-6fdb-48be-9648-3f6ac86aa33b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd"
Jul 16 20:28:57.849560 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.849522    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/772ee840-6fdb-48be-9648-3f6ac86aa33b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd\" (UID: \"772ee840-6fdb-48be-9648-3f6ac86aa33b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd"
Jul 16 20:28:57.850126 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.850107    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/772ee840-6fdb-48be-9648-3f6ac86aa33b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd\" (UID: \"772ee840-6fdb-48be-9648-3f6ac86aa33b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd"
Jul 16 20:28:57.958550 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.958412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb" event={"ID":"92c485e0-89bc-4c09-8a4e-c63823a79d5d","Type":"ContainerStarted","Data":"285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c"}
Jul 16 20:28:57.958708 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.958552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:28:57.978532 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.978478    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb" podStartSLOduration=2.759700099 podStartE2EDuration="5.978460927s" totalImagesPullingTime="3.218760828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:28:57.975651809 +0000 UTC m=+1702.123747240" watchObservedRunningTime="2026-07-16 20:28:57.978460927 +0000 UTC m=+1702.126556350"
Jul 16 20:28:57.992428 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:57.992397    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd"
Jul 16 20:28:58.127515 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:58.127485    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd"]
Jul 16 20:28:58.128812 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:28:58.128782    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod772ee840_6fdb_48be_9648_3f6ac86aa33b.slice/crio-921f8a862d66be43ee1e44f173d2b3643d2d41d51877b94e6f86a844acc0bc0e WatchSource:0}: Error finding container 921f8a862d66be43ee1e44f173d2b3643d2d41d51877b94e6f86a844acc0bc0e: Status 404 returned error can't find the container with id 921f8a862d66be43ee1e44f173d2b3643d2d41d51877b94e6f86a844acc0bc0e
Jul 16 20:28:58.964450 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:28:58.964402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd" event={"ID":"772ee840-6fdb-48be-9648-3f6ac86aa33b","Type":"ContainerStarted","Data":"921f8a862d66be43ee1e44f173d2b3643d2d41d51877b94e6f86a844acc0bc0e"}
Jul 16 20:29:02.618446 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:02.618405    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:29:03.970290 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:03.970258    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:29:16.056888 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:16.056842    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd" event={"ID":"772ee840-6fdb-48be-9648-3f6ac86aa33b","Type":"ContainerStarted","Data":"9199a488c82306e788660786ec5404932acffa2e84d16c35e285601d9cbd83a3"}
Jul 16 20:29:16.074683 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:16.074631    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd" podStartSLOduration=1.860672331 podStartE2EDuration="19.074615462s" totalImagesPullingTime="17.213943131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:28:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:29:16.072185169 +0000 UTC m=+1720.220280613" watchObservedRunningTime="2026-07-16 20:29:16.074615462 +0000 UTC m=+1720.222710883"
Jul 16 20:29:17.063811 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:17.063777    2579 generic.go:356] "Generic (PLEG): container finished" podID="772ee840-6fdb-48be-9648-3f6ac86aa33b" containerID="9199a488c82306e788660786ec5404932acffa2e84d16c35e285601d9cbd83a3" exitCode=0
Jul 16 20:29:17.064221 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:17.063867    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd" event={"ID":"772ee840-6fdb-48be-9648-3f6ac86aa33b","Type":"ContainerDied","Data":"9199a488c82306e788660786ec5404932acffa2e84d16c35e285601d9cbd83a3"}
Jul 16 20:29:18.203149 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:18.203123    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd"
Jul 16 20:29:18.340955 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:18.340854    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/772ee840-6fdb-48be-9648-3f6ac86aa33b-maas-api-service-ca\") pod \"772ee840-6fdb-48be-9648-3f6ac86aa33b\" (UID: \"772ee840-6fdb-48be-9648-3f6ac86aa33b\") "
Jul 16 20:29:18.341243 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:18.341217    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/772ee840-6fdb-48be-9648-3f6ac86aa33b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "772ee840-6fdb-48be-9648-3f6ac86aa33b" (UID: "772ee840-6fdb-48be-9648-3f6ac86aa33b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:29:18.442494 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:18.442459    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/772ee840-6fdb-48be-9648-3f6ac86aa33b-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:29:19.074937 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:19.074905    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd"
Jul 16 20:29:19.075085 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:19.074904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd" event={"ID":"772ee840-6fdb-48be-9648-3f6ac86aa33b","Type":"ContainerDied","Data":"921f8a862d66be43ee1e44f173d2b3643d2d41d51877b94e6f86a844acc0bc0e"}
Jul 16 20:29:19.075085 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:19.075032    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="921f8a862d66be43ee1e44f173d2b3643d2d41d51877b94e6f86a844acc0bc0e"
Jul 16 20:29:28.110762 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.110724    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"]
Jul 16 20:29:28.111268 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.111004    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb" podUID="92c485e0-89bc-4c09-8a4e-c63823a79d5d" containerName="maas-api" containerID="cri-o://285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c" gracePeriod=30
Jul 16 20:29:28.359936 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.359908    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:29:28.532048 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.532012    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcqqm\" (UniqueName: \"kubernetes.io/projected/92c485e0-89bc-4c09-8a4e-c63823a79d5d-kube-api-access-lcqqm\") pod \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") "
Jul 16 20:29:28.532250 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.532105    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c485e0-89bc-4c09-8a4e-c63823a79d5d-maas-api-tls\") pod \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") "
Jul 16 20:29:28.532250 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.532165    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c485e0-89bc-4c09-8a4e-c63823a79d5d-openshift-service-ca\") pod \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\" (UID: \"92c485e0-89bc-4c09-8a4e-c63823a79d5d\") "
Jul 16 20:29:28.532569 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.532547    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92c485e0-89bc-4c09-8a4e-c63823a79d5d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92c485e0-89bc-4c09-8a4e-c63823a79d5d" (UID: "92c485e0-89bc-4c09-8a4e-c63823a79d5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:29:28.534281 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.534250    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c485e0-89bc-4c09-8a4e-c63823a79d5d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92c485e0-89bc-4c09-8a4e-c63823a79d5d" (UID: "92c485e0-89bc-4c09-8a4e-c63823a79d5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:29:28.534384 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.534287    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c485e0-89bc-4c09-8a4e-c63823a79d5d-kube-api-access-lcqqm" (OuterVolumeSpecName: "kube-api-access-lcqqm") pod "92c485e0-89bc-4c09-8a4e-c63823a79d5d" (UID: "92c485e0-89bc-4c09-8a4e-c63823a79d5d"). InnerVolumeSpecName "kube-api-access-lcqqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:29:28.633970 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.633929    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcqqm\" (UniqueName: \"kubernetes.io/projected/92c485e0-89bc-4c09-8a4e-c63823a79d5d-kube-api-access-lcqqm\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:29:28.633970 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.633962    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c485e0-89bc-4c09-8a4e-c63823a79d5d-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:29:28.633970 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:28.633972    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c485e0-89bc-4c09-8a4e-c63823a79d5d-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:29:29.129600 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:29.129558    2579 generic.go:356] "Generic (PLEG): container finished" podID="92c485e0-89bc-4c09-8a4e-c63823a79d5d" containerID="285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c" exitCode=0
Jul 16 20:29:29.130271 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:29.129608    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"
Jul 16 20:29:29.130271 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:29.129641    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb" event={"ID":"92c485e0-89bc-4c09-8a4e-c63823a79d5d","Type":"ContainerDied","Data":"285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c"}
Jul 16 20:29:29.130271 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:29.129701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb" event={"ID":"92c485e0-89bc-4c09-8a4e-c63823a79d5d","Type":"ContainerDied","Data":"f2ec744277ac096a33809c33aea5e1a0c02a43debc9e23d34624a772a67f24f7"}
Jul 16 20:29:29.130271 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:29.129723    2579 scope.go:117] "RemoveContainer" containerID="285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c"
Jul 16 20:29:29.139065 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:29.139044    2579 scope.go:117] "RemoveContainer" containerID="285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c"
Jul 16 20:29:29.139357 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:29:29.139338    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c\": container with ID starting with 285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c not found: ID does not exist" containerID="285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c"
Jul 16 20:29:29.139409 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:29.139371    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c"} err="failed to get container status \"285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c\": rpc error: code = NotFound desc = could not find container \"285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c\": container with ID starting with 285b68af6d07f4059e68cff33fbf12ebd4c2ac3955899f9015cfa4444091185c not found: ID does not exist"
Jul 16 20:29:29.150568 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:29.150534    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"]
Jul 16 20:29:29.154683 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:29.154658    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-49aaf8ce-86d4c44cf7-wwxcb"]
Jul 16 20:29:30.404208 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:30.404172    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92c485e0-89bc-4c09-8a4e-c63823a79d5d" path="/var/lib/kubelet/pods/92c485e0-89bc-4c09-8a4e-c63823a79d5d/volumes"
Jul 16 20:29:31.381151 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:31.381114    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:29:45.090963 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.090924    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"]
Jul 16 20:29:45.091576 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.091552    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="772ee840-6fdb-48be-9648-3f6ac86aa33b" containerName="revoke-keys"
Jul 16 20:29:45.091656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.091579    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="772ee840-6fdb-48be-9648-3f6ac86aa33b" containerName="revoke-keys"
Jul 16 20:29:45.091656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.091598    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c485e0-89bc-4c09-8a4e-c63823a79d5d" containerName="maas-api"
Jul 16 20:29:45.091656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.091607    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c485e0-89bc-4c09-8a4e-c63823a79d5d" containerName="maas-api"
Jul 16 20:29:45.091824 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.091722    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c485e0-89bc-4c09-8a4e-c63823a79d5d" containerName="maas-api"
Jul 16 20:29:45.091824 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.091737    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="772ee840-6fdb-48be-9648-3f6ac86aa33b" containerName="revoke-keys"
Jul 16 20:29:45.095144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.095122    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:45.098751 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.098677    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-fa642852-serving-cert\""
Jul 16 20:29:45.098751 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.098700    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 20:29:45.098751 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.098705    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g78x2\""
Jul 16 20:29:45.099494 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.099477    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 20:29:45.106827 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.106791    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"]
Jul 16 20:29:45.180052 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.180014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc8d5\" (UniqueName: \"kubernetes.io/projected/4baf068c-6bbb-4eac-88fa-20c8f2138924-kube-api-access-cc8d5\") pod \"maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:45.180260 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.180069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf068c-6bbb-4eac-88fa-20c8f2138924-maas-api-tls\") pod \"maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:45.180260 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.180140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf068c-6bbb-4eac-88fa-20c8f2138924-openshift-service-ca\") pod \"maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:45.280627 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.280590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc8d5\" (UniqueName: \"kubernetes.io/projected/4baf068c-6bbb-4eac-88fa-20c8f2138924-kube-api-access-cc8d5\") pod \"maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:45.280844 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.280642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf068c-6bbb-4eac-88fa-20c8f2138924-maas-api-tls\") pod \"maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:45.280844 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.280667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf068c-6bbb-4eac-88fa-20c8f2138924-openshift-service-ca\") pod \"maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:45.280844 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:29:45.280765    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-serving-cert: secret "maas-api-e2e-migrate-fa642852-serving-cert" not found
Jul 16 20:29:45.281092 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:29:45.280854    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4baf068c-6bbb-4eac-88fa-20c8f2138924-maas-api-tls podName:4baf068c-6bbb-4eac-88fa-20c8f2138924 nodeName:}" failed. No retries permitted until 2026-07-16 20:29:45.780830987 +0000 UTC m=+1749.928926392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4baf068c-6bbb-4eac-88fa-20c8f2138924-maas-api-tls") pod "maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj" (UID: "4baf068c-6bbb-4eac-88fa-20c8f2138924") : secret "maas-api-e2e-migrate-fa642852-serving-cert" not found
Jul 16 20:29:45.281305 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.281280    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf068c-6bbb-4eac-88fa-20c8f2138924-openshift-service-ca\") pod \"maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:45.289369 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.289350    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc8d5\" (UniqueName: \"kubernetes.io/projected/4baf068c-6bbb-4eac-88fa-20c8f2138924-kube-api-access-cc8d5\") pod \"maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:45.786884 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.786848    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf068c-6bbb-4eac-88fa-20c8f2138924-maas-api-tls\") pod \"maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:45.789261 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:45.789237    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf068c-6bbb-4eac-88fa-20c8f2138924-maas-api-tls\") pod \"maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:46.006777 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:46.006739    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:46.143373 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:46.143346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"]
Jul 16 20:29:46.145680 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:29:46.145651    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4baf068c_6bbb_4eac_88fa_20c8f2138924.slice/crio-50f7001bb8f701f3cb546f6c5c0c79c74df6623566b79b4fa7d17d97665c72e3 WatchSource:0}: Error finding container 50f7001bb8f701f3cb546f6c5c0c79c74df6623566b79b4fa7d17d97665c72e3: Status 404 returned error can't find the container with id 50f7001bb8f701f3cb546f6c5c0c79c74df6623566b79b4fa7d17d97665c72e3
Jul 16 20:29:46.225691 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:46.225654    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj" event={"ID":"4baf068c-6bbb-4eac-88fa-20c8f2138924","Type":"ContainerStarted","Data":"50f7001bb8f701f3cb546f6c5c0c79c74df6623566b79b4fa7d17d97665c72e3"}
Jul 16 20:29:47.232980 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:47.232943    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj" event={"ID":"4baf068c-6bbb-4eac-88fa-20c8f2138924","Type":"ContainerStarted","Data":"fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59"}
Jul 16 20:29:47.233424 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:47.233017    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:47.252309 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:47.252251    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj" podStartSLOduration=1.91421292 podStartE2EDuration="2.252234237s" totalImagesPullingTime="338.021317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:29:47.250577086 +0000 UTC m=+1751.398672530" watchObservedRunningTime="2026-07-16 20:29:47.252234237 +0000 UTC m=+1751.400329752"
Jul 16 20:29:49.283400 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:49.283363    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"]
Jul 16 20:29:49.287168 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:49.287141    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"
Jul 16 20:29:49.289490 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:49.289466    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:29:49.294337 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:49.294307    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"]
Jul 16 20:29:49.318506 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:49.318461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87ac70c-b59b-4e13-b17d-663e2668bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fa642852-4bldp\" (UID: \"a87ac70c-b59b-4e13-b17d-663e2668bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"
Jul 16 20:29:49.420079 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:49.420019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87ac70c-b59b-4e13-b17d-663e2668bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fa642852-4bldp\" (UID: \"a87ac70c-b59b-4e13-b17d-663e2668bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"
Jul 16 20:29:49.420710 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:49.420689    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87ac70c-b59b-4e13-b17d-663e2668bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fa642852-4bldp\" (UID: \"a87ac70c-b59b-4e13-b17d-663e2668bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"
Jul 16 20:29:49.600543 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:49.600452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"
Jul 16 20:29:49.731565 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:49.731537    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"]
Jul 16 20:29:49.733046 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:29:49.733009    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda87ac70c_b59b_4e13_b17d_663e2668bd01.slice/crio-930897b4601e58ec86ee1cf0e20bd6050a72049de6fed0156e27a75dbe23b40e WatchSource:0}: Error finding container 930897b4601e58ec86ee1cf0e20bd6050a72049de6fed0156e27a75dbe23b40e: Status 404 returned error can't find the container with id 930897b4601e58ec86ee1cf0e20bd6050a72049de6fed0156e27a75dbe23b40e
Jul 16 20:29:50.252645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:50.252602    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp" event={"ID":"a87ac70c-b59b-4e13-b17d-663e2668bd01","Type":"ContainerStarted","Data":"0d3d050e764a4719a71bb7adefa25ea665400c6b968edd194739ec19cdf332ee"}
Jul 16 20:29:50.252799 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:50.252657    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp" event={"ID":"a87ac70c-b59b-4e13-b17d-663e2668bd01","Type":"ContainerStarted","Data":"930897b4601e58ec86ee1cf0e20bd6050a72049de6fed0156e27a75dbe23b40e"}
Jul 16 20:29:50.268636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:50.268587    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp" podStartSLOduration=1.268572377 podStartE2EDuration="1.268572377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:29:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:29:50.266363223 +0000 UTC m=+1754.414458647" watchObservedRunningTime="2026-07-16 20:29:50.268572377 +0000 UTC m=+1754.416667798"
Jul 16 20:29:51.258679 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:51.258644    2579 generic.go:356] "Generic (PLEG): container finished" podID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerID="0d3d050e764a4719a71bb7adefa25ea665400c6b968edd194739ec19cdf332ee" exitCode=7
Jul 16 20:29:51.259082 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:51.258734    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp" event={"ID":"a87ac70c-b59b-4e13-b17d-663e2668bd01","Type":"ContainerDied","Data":"0d3d050e764a4719a71bb7adefa25ea665400c6b968edd194739ec19cdf332ee"}
Jul 16 20:29:51.259082 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:51.258926    2579 scope.go:117] "RemoveContainer" containerID="0d3d050e764a4719a71bb7adefa25ea665400c6b968edd194739ec19cdf332ee"
Jul 16 20:29:52.265901 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:52.265808    2579 generic.go:356] "Generic (PLEG): container finished" podID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerID="2ec6314842ec05685b8ed9b7327828bbf6db699b772b80d40c743dad2512e4fd" exitCode=7
Jul 16 20:29:52.265901 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:52.265884    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp" event={"ID":"a87ac70c-b59b-4e13-b17d-663e2668bd01","Type":"ContainerDied","Data":"2ec6314842ec05685b8ed9b7327828bbf6db699b772b80d40c743dad2512e4fd"}
Jul 16 20:29:52.266416 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:52.265976    2579 scope.go:117] "RemoveContainer" containerID="0d3d050e764a4719a71bb7adefa25ea665400c6b968edd194739ec19cdf332ee"
Jul 16 20:29:52.266416 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:52.266185    2579 scope.go:117] "RemoveContainer" containerID="2ec6314842ec05685b8ed9b7327828bbf6db699b772b80d40c743dad2512e4fd"
Jul 16 20:29:52.266564 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:29:52.266463    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-fa642852-4bldp_odh-ai-gateway-infra(a87ac70c-b59b-4e13-b17d-663e2668bd01)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01"
Jul 16 20:29:53.246265 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:53.246187    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:29:53.273216 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:53.273177    2579 scope.go:117] "RemoveContainer" containerID="2ec6314842ec05685b8ed9b7327828bbf6db699b772b80d40c743dad2512e4fd"
Jul 16 20:29:53.273656 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:29:53.273416    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-fa642852-4bldp_odh-ai-gateway-infra(a87ac70c-b59b-4e13-b17d-663e2668bd01)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01"
Jul 16 20:29:53.292490 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:29:53.292458    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:30:00.135690 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:00.135653    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw"]
Jul 16 20:30:00.140360 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:00.140335    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d"]
Jul 16 20:30:00.140537 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:00.140472    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw"
Jul 16 20:30:00.144119 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:00.144095    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d"
Jul 16 20:30:00.146714 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:00.146686    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d"]
Jul 16 20:30:00.152973 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:00.152946    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw"]
Jul 16 20:30:00.156708 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:00.156686    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw"
Jul 16 20:30:00.162228 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:00.162211    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d"
Jul 16 20:30:00.303162 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:00.303125    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw"]
Jul 16 20:30:00.304736 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:30:00.304701    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493130f1_723b_45af_b378_658ba494ae55.slice/crio-f54d062fcb257746255caa8840ad58ff78d25361bae2b73d2e8770f89afcfdc4 WatchSource:0}: Error finding container f54d062fcb257746255caa8840ad58ff78d25361bae2b73d2e8770f89afcfdc4: Status 404 returned error can't find the container with id f54d062fcb257746255caa8840ad58ff78d25361bae2b73d2e8770f89afcfdc4
Jul 16 20:30:00.321590 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:00.321563    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d"]
Jul 16 20:30:00.322672 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:30:00.322648    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd577682e_3e76_44c7_b0ac_cf0580cdd2d2.slice/crio-1e867f902aab0f35309c66b61addf18d7e12f0cc2a0fd7dbb52b875581e74d38 WatchSource:0}: Error finding container 1e867f902aab0f35309c66b61addf18d7e12f0cc2a0fd7dbb52b875581e74d38: Status 404 returned error can't find the container with id 1e867f902aab0f35309c66b61addf18d7e12f0cc2a0fd7dbb52b875581e74d38
Jul 16 20:30:01.317240 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:01.317203    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d" event={"ID":"d577682e-3e76-44c7-b0ac-cf0580cdd2d2","Type":"ContainerStarted","Data":"b72de3f5ced1979c01aa3556eaa05d607017b1205119dbfce1d11b48163a2428"}
Jul 16 20:30:01.317703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:01.317267    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d" event={"ID":"d577682e-3e76-44c7-b0ac-cf0580cdd2d2","Type":"ContainerStarted","Data":"1e867f902aab0f35309c66b61addf18d7e12f0cc2a0fd7dbb52b875581e74d38"}
Jul 16 20:30:01.318669 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:01.318643    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw" event={"ID":"493130f1-723b-45af-b378-658ba494ae55","Type":"ContainerStarted","Data":"8563ce99aa230264533b6178f151a247b394d5c233e441a93d1ecae213f940d7"}
Jul 16 20:30:01.318804 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:01.318685    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw" event={"ID":"493130f1-723b-45af-b378-658ba494ae55","Type":"ContainerStarted","Data":"f54d062fcb257746255caa8840ad58ff78d25361bae2b73d2e8770f89afcfdc4"}
Jul 16 20:30:01.331132 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:01.331077    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d" podStartSLOduration=1.331060392 podStartE2EDuration="1.331060392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:30:01.330662517 +0000 UTC m=+1765.478757940" watchObservedRunningTime="2026-07-16 20:30:01.331060392 +0000 UTC m=+1765.479155808"
Jul 16 20:30:01.354130 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:01.349389    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw" podStartSLOduration=1.34937062 podStartE2EDuration="1.34937062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:30:01.345865158 +0000 UTC m=+1765.493960581" watchObservedRunningTime="2026-07-16 20:30:01.34937062 +0000 UTC m=+1765.497466044"
Jul 16 20:30:02.325116 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:02.325077    2579 generic.go:356] "Generic (PLEG): container finished" podID="d577682e-3e76-44c7-b0ac-cf0580cdd2d2" containerID="b72de3f5ced1979c01aa3556eaa05d607017b1205119dbfce1d11b48163a2428" exitCode=0
Jul 16 20:30:02.325616 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:02.325168    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d" event={"ID":"d577682e-3e76-44c7-b0ac-cf0580cdd2d2","Type":"ContainerDied","Data":"b72de3f5ced1979c01aa3556eaa05d607017b1205119dbfce1d11b48163a2428"}
Jul 16 20:30:02.326634 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:02.326611    2579 generic.go:356] "Generic (PLEG): container finished" podID="493130f1-723b-45af-b378-658ba494ae55" containerID="8563ce99aa230264533b6178f151a247b394d5c233e441a93d1ecae213f940d7" exitCode=0
Jul 16 20:30:02.326764 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:02.326637    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw" event={"ID":"493130f1-723b-45af-b378-658ba494ae55","Type":"ContainerDied","Data":"8563ce99aa230264533b6178f151a247b394d5c233e441a93d1ecae213f940d7"}
Jul 16 20:30:03.502119 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:03.502084    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw"
Jul 16 20:30:03.504630 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:03.504611    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d"
Jul 16 20:30:04.339909 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:04.339882    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d"
Jul 16 20:30:04.340106 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:04.339880    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737230-7qf2d" event={"ID":"d577682e-3e76-44c7-b0ac-cf0580cdd2d2","Type":"ContainerDied","Data":"1e867f902aab0f35309c66b61addf18d7e12f0cc2a0fd7dbb52b875581e74d38"}
Jul 16 20:30:04.340106 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:04.340001    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e867f902aab0f35309c66b61addf18d7e12f0cc2a0fd7dbb52b875581e74d38"
Jul 16 20:30:04.341370 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:04.341351    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw"
Jul 16 20:30:04.341536 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:04.341403    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw" event={"ID":"493130f1-723b-45af-b378-658ba494ae55","Type":"ContainerDied","Data":"f54d062fcb257746255caa8840ad58ff78d25361bae2b73d2e8770f89afcfdc4"}
Jul 16 20:30:04.341536 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:04.341464    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f54d062fcb257746255caa8840ad58ff78d25361bae2b73d2e8770f89afcfdc4"
Jul 16 20:30:07.398346 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:07.398320    2579 scope.go:117] "RemoveContainer" containerID="2ec6314842ec05685b8ed9b7327828bbf6db699b772b80d40c743dad2512e4fd"
Jul 16 20:30:08.365529 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:08.365418    2579 generic.go:356] "Generic (PLEG): container finished" podID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerID="0e5d087f1a3e61b3e5f623a427ab058d8aeac2e6050eadf916eee7aedbdc18ab" exitCode=0
Jul 16 20:30:08.365529 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:08.365493    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp" event={"ID":"a87ac70c-b59b-4e13-b17d-663e2668bd01","Type":"ContainerDied","Data":"0e5d087f1a3e61b3e5f623a427ab058d8aeac2e6050eadf916eee7aedbdc18ab"}
Jul 16 20:30:08.365709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:08.365591    2579 scope.go:117] "RemoveContainer" containerID="2ec6314842ec05685b8ed9b7327828bbf6db699b772b80d40c743dad2512e4fd"
Jul 16 20:30:09.397748 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:09.397710    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"]
Jul 16 20:30:09.503068 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:09.503045    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"
Jul 16 20:30:09.622855 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:09.622760    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87ac70c-b59b-4e13-b17d-663e2668bd01-maas-api-service-ca\") pod \"a87ac70c-b59b-4e13-b17d-663e2668bd01\" (UID: \"a87ac70c-b59b-4e13-b17d-663e2668bd01\") "
Jul 16 20:30:09.623147 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:09.623126    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a87ac70c-b59b-4e13-b17d-663e2668bd01-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a87ac70c-b59b-4e13-b17d-663e2668bd01" (UID: "a87ac70c-b59b-4e13-b17d-663e2668bd01"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:30:09.723817 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:09.723770    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a87ac70c-b59b-4e13-b17d-663e2668bd01-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:30:10.379963 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:10.379932    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"
Jul 16 20:30:10.380179 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:10.379968    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp" event={"ID":"a87ac70c-b59b-4e13-b17d-663e2668bd01","Type":"ContainerDied","Data":"930897b4601e58ec86ee1cf0e20bd6050a72049de6fed0156e27a75dbe23b40e"}
Jul 16 20:30:10.380179 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:10.380019    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="930897b4601e58ec86ee1cf0e20bd6050a72049de6fed0156e27a75dbe23b40e"
Jul 16 20:30:10.420490 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:10.420457    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"]
Jul 16 20:30:10.428879 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:10.428850    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-4bldp"]
Jul 16 20:30:12.409994 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:12.409955    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01" path="/var/lib/kubelet/pods/a87ac70c-b59b-4e13-b17d-663e2668bd01/volumes"
Jul 16 20:30:49.334583 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.334540    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx"]
Jul 16 20:30:49.335948 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.335917    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="493130f1-723b-45af-b378-658ba494ae55" containerName="cleanup"
Jul 16 20:30:49.336109 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.336098    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="493130f1-723b-45af-b378-658ba494ae55" containerName="cleanup"
Jul 16 20:30:49.336222 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.336210    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerName="revoke-keys"
Jul 16 20:30:49.336296 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.336287    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerName="revoke-keys"
Jul 16 20:30:49.336360 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.336352    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerName="revoke-keys"
Jul 16 20:30:49.336423 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.336415    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerName="revoke-keys"
Jul 16 20:30:49.336537 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.336526    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerName="revoke-keys"
Jul 16 20:30:49.336601 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.336593    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerName="revoke-keys"
Jul 16 20:30:49.336687 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.336677    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d577682e-3e76-44c7-b0ac-cf0580cdd2d2" containerName="cleanup"
Jul 16 20:30:49.336751 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.336743    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d577682e-3e76-44c7-b0ac-cf0580cdd2d2" containerName="cleanup"
Jul 16 20:30:49.336996 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.336981    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerName="revoke-keys"
Jul 16 20:30:49.337093 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.337084    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="493130f1-723b-45af-b378-658ba494ae55" containerName="cleanup"
Jul 16 20:30:49.337163 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.337154    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerName="revoke-keys"
Jul 16 20:30:49.337230 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.337222    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d577682e-3e76-44c7-b0ac-cf0580cdd2d2" containerName="cleanup"
Jul 16 20:30:49.341735 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.341708    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx"
Jul 16 20:30:49.344081 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.344054    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:30:49.344212 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.344140    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx"]
Jul 16 20:30:49.488784 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.488746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx\" (UID: \"ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx"
Jul 16 20:30:49.589668 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.589585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx\" (UID: \"ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx"
Jul 16 20:30:49.590184 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.590162    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx\" (UID: \"ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx"
Jul 16 20:30:49.654549 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.654510    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx"
Jul 16 20:30:49.792597 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:49.792565    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx"]
Jul 16 20:30:49.795764 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:30:49.795740    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb7ba31_08d4_4c86_a5a0_a7209ac1e65d.slice/crio-09fe51b02f40868090182144a325a0dc5eba573b5cbcdf87c61dd5e49dfb0e17 WatchSource:0}: Error finding container 09fe51b02f40868090182144a325a0dc5eba573b5cbcdf87c61dd5e49dfb0e17: Status 404 returned error can't find the container with id 09fe51b02f40868090182144a325a0dc5eba573b5cbcdf87c61dd5e49dfb0e17
Jul 16 20:30:50.593112 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:50.593073    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx" event={"ID":"ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d","Type":"ContainerStarted","Data":"67f861d91fbaf631cf24e0501be896adc4d927e5ec8606248f1e4f107367f30b"}
Jul 16 20:30:50.593521 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:50.593132    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx" event={"ID":"ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d","Type":"ContainerStarted","Data":"09fe51b02f40868090182144a325a0dc5eba573b5cbcdf87c61dd5e49dfb0e17"}
Jul 16 20:30:50.623598 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:50.623549    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx" podStartSLOduration=1.6235344280000001 podStartE2EDuration="1.623534428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:30:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:30:50.622506819 +0000 UTC m=+1814.770602239" watchObservedRunningTime="2026-07-16 20:30:50.623534428 +0000 UTC m=+1814.771629847"
Jul 16 20:30:51.600232 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:51.600194    2579 generic.go:356] "Generic (PLEG): container finished" podID="ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d" containerID="67f861d91fbaf631cf24e0501be896adc4d927e5ec8606248f1e4f107367f30b" exitCode=0
Jul 16 20:30:51.600684 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:51.600277    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx" event={"ID":"ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d","Type":"ContainerDied","Data":"67f861d91fbaf631cf24e0501be896adc4d927e5ec8606248f1e4f107367f30b"}
Jul 16 20:30:52.691558 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:52.691522    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:30:52.727830 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:52.727805    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx"
Jul 16 20:30:52.922925 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:52.922886    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d-maas-api-service-ca\") pod \"ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d\" (UID: \"ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d\") "
Jul 16 20:30:52.923290 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:52.923259    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d" (UID: "ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:30:53.024519 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:53.024479    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:30:53.613070 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:53.613029    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx" event={"ID":"ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d","Type":"ContainerDied","Data":"09fe51b02f40868090182144a325a0dc5eba573b5cbcdf87c61dd5e49dfb0e17"}
Jul 16 20:30:53.613070 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:53.613063    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx"
Jul 16 20:30:53.613290 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:53.613086    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09fe51b02f40868090182144a325a0dc5eba573b5cbcdf87c61dd5e49dfb0e17"
Jul 16 20:30:54.900021 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:54.899986    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"]
Jul 16 20:30:54.900492 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:54.900193    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj" podUID="4baf068c-6bbb-4eac-88fa-20c8f2138924" containerName="maas-api" containerID="cri-o://fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59" gracePeriod=30
Jul 16 20:30:54.978711 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:54.978622    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw"]
Jul 16 20:30:54.993514 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:54.993481    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-fa642852-29737230-sbtnw"]
Jul 16 20:30:55.148053 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.148023    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:30:55.240716 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.240616    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc8d5\" (UniqueName: \"kubernetes.io/projected/4baf068c-6bbb-4eac-88fa-20c8f2138924-kube-api-access-cc8d5\") pod \"4baf068c-6bbb-4eac-88fa-20c8f2138924\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") "
Jul 16 20:30:55.240716 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.240688    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf068c-6bbb-4eac-88fa-20c8f2138924-maas-api-tls\") pod \"4baf068c-6bbb-4eac-88fa-20c8f2138924\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") "
Jul 16 20:30:55.240911 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.240732    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf068c-6bbb-4eac-88fa-20c8f2138924-openshift-service-ca\") pod \"4baf068c-6bbb-4eac-88fa-20c8f2138924\" (UID: \"4baf068c-6bbb-4eac-88fa-20c8f2138924\") "
Jul 16 20:30:55.241100 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.241075    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4baf068c-6bbb-4eac-88fa-20c8f2138924-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4baf068c-6bbb-4eac-88fa-20c8f2138924" (UID: "4baf068c-6bbb-4eac-88fa-20c8f2138924"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:30:55.242836 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.242804    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4baf068c-6bbb-4eac-88fa-20c8f2138924-kube-api-access-cc8d5" (OuterVolumeSpecName: "kube-api-access-cc8d5") pod "4baf068c-6bbb-4eac-88fa-20c8f2138924" (UID: "4baf068c-6bbb-4eac-88fa-20c8f2138924"). InnerVolumeSpecName "kube-api-access-cc8d5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:30:55.242952 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.242834    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4baf068c-6bbb-4eac-88fa-20c8f2138924-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4baf068c-6bbb-4eac-88fa-20c8f2138924" (UID: "4baf068c-6bbb-4eac-88fa-20c8f2138924"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:30:55.341850 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.341815    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc8d5\" (UniqueName: \"kubernetes.io/projected/4baf068c-6bbb-4eac-88fa-20c8f2138924-kube-api-access-cc8d5\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:30:55.341850 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.341845    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4baf068c-6bbb-4eac-88fa-20c8f2138924-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:30:55.341850 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.341855    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4baf068c-6bbb-4eac-88fa-20c8f2138924-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:30:55.624855 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.624755    2579 generic.go:356] "Generic (PLEG): container finished" podID="4baf068c-6bbb-4eac-88fa-20c8f2138924" containerID="fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59" exitCode=0
Jul 16 20:30:55.624855 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.624808    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"
Jul 16 20:30:55.625085 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.624810    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj" event={"ID":"4baf068c-6bbb-4eac-88fa-20c8f2138924","Type":"ContainerDied","Data":"fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59"}
Jul 16 20:30:55.625085 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.624935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj" event={"ID":"4baf068c-6bbb-4eac-88fa-20c8f2138924","Type":"ContainerDied","Data":"50f7001bb8f701f3cb546f6c5c0c79c74df6623566b79b4fa7d17d97665c72e3"}
Jul 16 20:30:55.625085 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.624964    2579 scope.go:117] "RemoveContainer" containerID="fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59"
Jul 16 20:30:55.634870 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.634853    2579 scope.go:117] "RemoveContainer" containerID="fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59"
Jul 16 20:30:55.635121 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:30:55.635103    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59\": container with ID starting with fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59 not found: ID does not exist" containerID="fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59"
Jul 16 20:30:55.635186 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.635132    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59"} err="failed to get container status \"fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59\": rpc error: code = NotFound desc = could not find container \"fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59\": container with ID starting with fa57f25f214ddcaf6ef96324c6660ecdc9b14f94b279a4747b693011109adb59 not found: ID does not exist"
Jul 16 20:30:55.647264 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.647233    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"]
Jul 16 20:30:55.651494 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:55.651465    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fa642852-586cf7dcc9-pwhdj"]
Jul 16 20:30:56.404263 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:56.404186    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="493130f1-723b-45af-b378-658ba494ae55" path="/var/lib/kubelet/pods/493130f1-723b-45af-b378-658ba494ae55/volumes"
Jul 16 20:30:56.404637 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:30:56.404557    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4baf068c-6bbb-4eac-88fa-20c8f2138924" path="/var/lib/kubelet/pods/4baf068c-6bbb-4eac-88fa-20c8f2138924/volumes"
Jul 16 20:31:03.278134 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.278096    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"]
Jul 16 20:31:03.278610 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.278590    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4baf068c-6bbb-4eac-88fa-20c8f2138924" containerName="maas-api"
Jul 16 20:31:03.278610 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.278611    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4baf068c-6bbb-4eac-88fa-20c8f2138924" containerName="maas-api"
Jul 16 20:31:03.278696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.278632    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d" containerName="revoke-keys"
Jul 16 20:31:03.278696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.278638    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d" containerName="revoke-keys"
Jul 16 20:31:03.278696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.278696    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4baf068c-6bbb-4eac-88fa-20c8f2138924" containerName="maas-api"
Jul 16 20:31:03.278802 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.278707    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d" containerName="revoke-keys"
Jul 16 20:31:03.278802 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.278714    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87ac70c-b59b-4e13-b17d-663e2668bd01" containerName="revoke-keys"
Jul 16 20:31:03.282942 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.282923    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:03.285227 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.285202    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e9ceb054-serving-cert\""
Jul 16 20:31:03.285518 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.285498    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 20:31:03.285591 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.285524    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g78x2\""
Jul 16 20:31:03.285974 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.285955    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 20:31:03.291239 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.291217    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"]
Jul 16 20:31:03.312075 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.312046    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17a3d412-b301-44e7-b55c-0f2a0805ad9e-maas-api-tls\") pod \"maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:03.312237 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.312103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17a3d412-b301-44e7-b55c-0f2a0805ad9e-openshift-service-ca\") pod \"maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:03.312237 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.312179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppff6\" (UniqueName: \"kubernetes.io/projected/17a3d412-b301-44e7-b55c-0f2a0805ad9e-kube-api-access-ppff6\") pod \"maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:03.413480 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.413418    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17a3d412-b301-44e7-b55c-0f2a0805ad9e-openshift-service-ca\") pod \"maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:03.413654 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.413508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppff6\" (UniqueName: \"kubernetes.io/projected/17a3d412-b301-44e7-b55c-0f2a0805ad9e-kube-api-access-ppff6\") pod \"maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:03.413654 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.413555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17a3d412-b301-44e7-b55c-0f2a0805ad9e-maas-api-tls\") pod \"maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:03.413724 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:31:03.413660    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-serving-cert: secret "maas-api-e2e-ait-e9ceb054-serving-cert" not found
Jul 16 20:31:03.413724 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:31:03.413722    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17a3d412-b301-44e7-b55c-0f2a0805ad9e-maas-api-tls podName:17a3d412-b301-44e7-b55c-0f2a0805ad9e nodeName:}" failed. No retries permitted until 2026-07-16 20:31:03.913706696 +0000 UTC m=+1828.061802096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/17a3d412-b301-44e7-b55c-0f2a0805ad9e-maas-api-tls") pod "maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7" (UID: "17a3d412-b301-44e7-b55c-0f2a0805ad9e") : secret "maas-api-e2e-ait-e9ceb054-serving-cert" not found
Jul 16 20:31:03.414171 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.414151    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17a3d412-b301-44e7-b55c-0f2a0805ad9e-openshift-service-ca\") pod \"maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:03.426824 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.426792    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppff6\" (UniqueName: \"kubernetes.io/projected/17a3d412-b301-44e7-b55c-0f2a0805ad9e-kube-api-access-ppff6\") pod \"maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:03.918258 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.918222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17a3d412-b301-44e7-b55c-0f2a0805ad9e-maas-api-tls\") pod \"maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:03.926972 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:03.920681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17a3d412-b301-44e7-b55c-0f2a0805ad9e-maas-api-tls\") pod \"maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:04.194734 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:04.194647    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:04.332640 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:04.332613    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"]
Jul 16 20:31:04.334287 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:31:04.334257    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17a3d412_b301_44e7_b55c_0f2a0805ad9e.slice/crio-56fb0f6f8959689e9637b462c4bd3034dd88bfba3d67e8dd1964de82e4b7b7b9 WatchSource:0}: Error finding container 56fb0f6f8959689e9637b462c4bd3034dd88bfba3d67e8dd1964de82e4b7b7b9: Status 404 returned error can't find the container with id 56fb0f6f8959689e9637b462c4bd3034dd88bfba3d67e8dd1964de82e4b7b7b9
Jul 16 20:31:04.675373 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:04.675333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7" event={"ID":"17a3d412-b301-44e7-b55c-0f2a0805ad9e","Type":"ContainerStarted","Data":"56fb0f6f8959689e9637b462c4bd3034dd88bfba3d67e8dd1964de82e4b7b7b9"}
Jul 16 20:31:05.682312 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:05.682279    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7" event={"ID":"17a3d412-b301-44e7-b55c-0f2a0805ad9e","Type":"ContainerStarted","Data":"ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3"}
Jul 16 20:31:05.682723 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:05.682353    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:05.702696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:05.702646    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7" podStartSLOduration=2.144869275 podStartE2EDuration="2.702630953s" totalImagesPullingTime="557.761678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:31:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:31:05.699644913 +0000 UTC m=+1829.847740335" watchObservedRunningTime="2026-07-16 20:31:05.702630953 +0000 UTC m=+1829.850726374"
Jul 16 20:31:08.125055 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:08.125020    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"]
Jul 16 20:31:08.128516 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:08.128496    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"
Jul 16 20:31:08.130561 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:08.130542    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:31:08.135629 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:08.135600    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"]
Jul 16 20:31:08.160322 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:08.160287    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41202d69-db47-4f43-b4c4-1a4c52013c6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q\" (UID: \"41202d69-db47-4f43-b4c4-1a4c52013c6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"
Jul 16 20:31:08.260998 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:08.260959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41202d69-db47-4f43-b4c4-1a4c52013c6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q\" (UID: \"41202d69-db47-4f43-b4c4-1a4c52013c6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"
Jul 16 20:31:08.261654 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:08.261626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41202d69-db47-4f43-b4c4-1a4c52013c6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q\" (UID: \"41202d69-db47-4f43-b4c4-1a4c52013c6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"
Jul 16 20:31:08.441378 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:08.441340    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"
Jul 16 20:31:08.570753 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:08.570722    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"]
Jul 16 20:31:08.574881 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:31:08.574853    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41202d69_db47_4f43_b4c4_1a4c52013c6e.slice/crio-c4c2c0457d4aae8f6cdfcb2e43a35b338d6ac9ca8e45fa7cbd2d56d154bbf231 WatchSource:0}: Error finding container c4c2c0457d4aae8f6cdfcb2e43a35b338d6ac9ca8e45fa7cbd2d56d154bbf231: Status 404 returned error can't find the container with id c4c2c0457d4aae8f6cdfcb2e43a35b338d6ac9ca8e45fa7cbd2d56d154bbf231
Jul 16 20:31:08.700321 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:08.700239    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q" event={"ID":"41202d69-db47-4f43-b4c4-1a4c52013c6e","Type":"ContainerStarted","Data":"c4c2c0457d4aae8f6cdfcb2e43a35b338d6ac9ca8e45fa7cbd2d56d154bbf231"}
Jul 16 20:31:09.707359 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:09.707326    2579 generic.go:356] "Generic (PLEG): container finished" podID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerID="ec0b1fd14f93c46fae4630cd76e6ccf96f21a6f78a9f94941a995a1776b4e356" exitCode=7
Jul 16 20:31:09.707770 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:09.707417    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q" event={"ID":"41202d69-db47-4f43-b4c4-1a4c52013c6e","Type":"ContainerDied","Data":"ec0b1fd14f93c46fae4630cd76e6ccf96f21a6f78a9f94941a995a1776b4e356"}
Jul 16 20:31:09.707770 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:09.707604    2579 scope.go:117] "RemoveContainer" containerID="ec0b1fd14f93c46fae4630cd76e6ccf96f21a6f78a9f94941a995a1776b4e356"
Jul 16 20:31:10.720847 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:10.720811    2579 generic.go:356] "Generic (PLEG): container finished" podID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerID="0b23ed082dadca3b4eb39fd4b57d6a35558214803e732807d843acb2f55f2ebe" exitCode=7
Jul 16 20:31:10.721311 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:10.720969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q" event={"ID":"41202d69-db47-4f43-b4c4-1a4c52013c6e","Type":"ContainerDied","Data":"0b23ed082dadca3b4eb39fd4b57d6a35558214803e732807d843acb2f55f2ebe"}
Jul 16 20:31:10.721311 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:10.721061    2579 scope.go:117] "RemoveContainer" containerID="ec0b1fd14f93c46fae4630cd76e6ccf96f21a6f78a9f94941a995a1776b4e356"
Jul 16 20:31:10.721311 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:10.721259    2579 scope.go:117] "RemoveContainer" containerID="0b23ed082dadca3b4eb39fd4b57d6a35558214803e732807d843acb2f55f2ebe"
Jul 16 20:31:10.721541 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:31:10.721520    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q_odh-ai-gateway-infra(41202d69-db47-4f43-b4c4-1a4c52013c6e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e"
Jul 16 20:31:11.694311 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:11.694280    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:31:11.730774 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:11.730748    2579 scope.go:117] "RemoveContainer" containerID="0b23ed082dadca3b4eb39fd4b57d6a35558214803e732807d843acb2f55f2ebe"
Jul 16 20:31:11.731157 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:31:11.730939    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q_odh-ai-gateway-infra(41202d69-db47-4f43-b4c4-1a4c52013c6e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e"
Jul 16 20:31:11.874990 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:11.874952    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:31:24.398981 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:24.398940    2579 scope.go:117] "RemoveContainer" containerID="0b23ed082dadca3b4eb39fd4b57d6a35558214803e732807d843acb2f55f2ebe"
Jul 16 20:31:24.804874 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:24.804839    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q" event={"ID":"41202d69-db47-4f43-b4c4-1a4c52013c6e","Type":"ContainerStarted","Data":"9b43f6c3d5c7600df332d2361102fb4bfc9f034fb689b615cbc223a6fbcd87b8"}
Jul 16 20:31:24.825077 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:24.825025    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q" podStartSLOduration=16.825008022 podStartE2EDuration="16.825008022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:31:24.822340368 +0000 UTC m=+1848.970435814" watchObservedRunningTime="2026-07-16 20:31:24.825008022 +0000 UTC m=+1848.973103444"
Jul 16 20:31:25.812661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:25.812625    2579 generic.go:356] "Generic (PLEG): container finished" podID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerID="9b43f6c3d5c7600df332d2361102fb4bfc9f034fb689b615cbc223a6fbcd87b8" exitCode=0
Jul 16 20:31:25.813084 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:25.812697    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q" event={"ID":"41202d69-db47-4f43-b4c4-1a4c52013c6e","Type":"ContainerDied","Data":"9b43f6c3d5c7600df332d2361102fb4bfc9f034fb689b615cbc223a6fbcd87b8"}
Jul 16 20:31:25.813084 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:25.812760    2579 scope.go:117] "RemoveContainer" containerID="0b23ed082dadca3b4eb39fd4b57d6a35558214803e732807d843acb2f55f2ebe"
Jul 16 20:31:25.835381 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:25.835351    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"]
Jul 16 20:31:26.952339 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:26.952306    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"
Jul 16 20:31:27.033528 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:27.033493    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41202d69-db47-4f43-b4c4-1a4c52013c6e-maas-api-service-ca\") pod \"41202d69-db47-4f43-b4c4-1a4c52013c6e\" (UID: \"41202d69-db47-4f43-b4c4-1a4c52013c6e\") "
Jul 16 20:31:27.033859 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:27.033835    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41202d69-db47-4f43-b4c4-1a4c52013c6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41202d69-db47-4f43-b4c4-1a4c52013c6e" (UID: "41202d69-db47-4f43-b4c4-1a4c52013c6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:31:27.134369 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:27.134282    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41202d69-db47-4f43-b4c4-1a4c52013c6e-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:31:27.826518 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:27.826480    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q" event={"ID":"41202d69-db47-4f43-b4c4-1a4c52013c6e","Type":"ContainerDied","Data":"c4c2c0457d4aae8f6cdfcb2e43a35b338d6ac9ca8e45fa7cbd2d56d154bbf231"}
Jul 16 20:31:27.826723 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:27.826539    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4c2c0457d4aae8f6cdfcb2e43a35b338d6ac9ca8e45fa7cbd2d56d154bbf231"
Jul 16 20:31:27.826723 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:27.826492    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"
Jul 16 20:31:27.888058 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:27.888021    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"]
Jul 16 20:31:27.898964 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:27.898929    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-cmb9q"]
Jul 16 20:31:28.403899 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:28.403861    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e" path="/var/lib/kubelet/pods/41202d69-db47-4f43-b4c4-1a4c52013c6e/volumes"
Jul 16 20:31:58.172507 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.172404    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt"]
Jul 16 20:31:58.173030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.172972    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerName="revoke-keys"
Jul 16 20:31:58.173030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.172994    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerName="revoke-keys"
Jul 16 20:31:58.173030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.173020    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerName="revoke-keys"
Jul 16 20:31:58.173030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.173030    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerName="revoke-keys"
Jul 16 20:31:58.173235 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.173134    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerName="revoke-keys"
Jul 16 20:31:58.173235 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.173147    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerName="revoke-keys"
Jul 16 20:31:58.176353 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.176330    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt"
Jul 16 20:31:58.179027 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.178847    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:31:58.181990 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.181965    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt"]
Jul 16 20:31:58.290228 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.290193    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/249c3330-2d24-4e47-ac22-31a48f8a0ea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt\" (UID: \"249c3330-2d24-4e47-ac22-31a48f8a0ea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt"
Jul 16 20:31:58.391775 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.391738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/249c3330-2d24-4e47-ac22-31a48f8a0ea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt\" (UID: \"249c3330-2d24-4e47-ac22-31a48f8a0ea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt"
Jul 16 20:31:58.392488 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.392462    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/249c3330-2d24-4e47-ac22-31a48f8a0ea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt\" (UID: \"249c3330-2d24-4e47-ac22-31a48f8a0ea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt"
Jul 16 20:31:58.488466 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.488325    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt"
Jul 16 20:31:58.623038 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:58.622988    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt"]
Jul 16 20:31:58.625305 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:31:58.625277    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod249c3330_2d24_4e47_ac22_31a48f8a0ea7.slice/crio-f7c69b50d488288ac6a582e7bdcd59e24d78951e8eb6078bd3c9d7bc4bee8742 WatchSource:0}: Error finding container f7c69b50d488288ac6a582e7bdcd59e24d78951e8eb6078bd3c9d7bc4bee8742: Status 404 returned error can't find the container with id f7c69b50d488288ac6a582e7bdcd59e24d78951e8eb6078bd3c9d7bc4bee8742
Jul 16 20:31:59.001237 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:59.001203    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt" event={"ID":"249c3330-2d24-4e47-ac22-31a48f8a0ea7","Type":"ContainerStarted","Data":"ded59b39efe2f67ef628eaf2bbd9bb7280d3c66c7f7dde5e53bbe94007b8bc7e"}
Jul 16 20:31:59.001418 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:59.001258    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt" event={"ID":"249c3330-2d24-4e47-ac22-31a48f8a0ea7","Type":"ContainerStarted","Data":"f7c69b50d488288ac6a582e7bdcd59e24d78951e8eb6078bd3c9d7bc4bee8742"}
Jul 16 20:31:59.016887 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:31:59.016835    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt" podStartSLOduration=1.016820079 podStartE2EDuration="1.016820079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:31:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:31:59.015727388 +0000 UTC m=+1883.163822831" watchObservedRunningTime="2026-07-16 20:31:59.016820079 +0000 UTC m=+1883.164915500"
Jul 16 20:32:00.008884 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:00.008850    2579 generic.go:356] "Generic (PLEG): container finished" podID="249c3330-2d24-4e47-ac22-31a48f8a0ea7" containerID="ded59b39efe2f67ef628eaf2bbd9bb7280d3c66c7f7dde5e53bbe94007b8bc7e" exitCode=0
Jul 16 20:32:00.009326 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:00.008929    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt" event={"ID":"249c3330-2d24-4e47-ac22-31a48f8a0ea7","Type":"ContainerDied","Data":"ded59b39efe2f67ef628eaf2bbd9bb7280d3c66c7f7dde5e53bbe94007b8bc7e"}
Jul 16 20:32:01.169554 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:01.169521    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt"
Jul 16 20:32:01.215842 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:01.215811    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/249c3330-2d24-4e47-ac22-31a48f8a0ea7-maas-api-service-ca\") pod \"249c3330-2d24-4e47-ac22-31a48f8a0ea7\" (UID: \"249c3330-2d24-4e47-ac22-31a48f8a0ea7\") "
Jul 16 20:32:01.216120 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:01.216098    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/249c3330-2d24-4e47-ac22-31a48f8a0ea7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "249c3330-2d24-4e47-ac22-31a48f8a0ea7" (UID: "249c3330-2d24-4e47-ac22-31a48f8a0ea7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:32:01.316752 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:01.316652    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/249c3330-2d24-4e47-ac22-31a48f8a0ea7-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:32:02.020493 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:02.020461    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt" event={"ID":"249c3330-2d24-4e47-ac22-31a48f8a0ea7","Type":"ContainerDied","Data":"f7c69b50d488288ac6a582e7bdcd59e24d78951e8eb6078bd3c9d7bc4bee8742"}
Jul 16 20:32:02.020673 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:02.020499    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt"
Jul 16 20:32:02.020673 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:02.020509    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7c69b50d488288ac6a582e7bdcd59e24d78951e8eb6078bd3c9d7bc4bee8742"
Jul 16 20:32:08.583416 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.583375    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"]
Jul 16 20:32:08.583947 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.583649    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7" podUID="17a3d412-b301-44e7-b55c-0f2a0805ad9e" containerName="maas-api" containerID="cri-o://ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3" gracePeriod=30
Jul 16 20:32:08.832771 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.832741    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:32:08.885094 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.885014    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17a3d412-b301-44e7-b55c-0f2a0805ad9e-maas-api-tls\") pod \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") "
Jul 16 20:32:08.885094 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.885084    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17a3d412-b301-44e7-b55c-0f2a0805ad9e-openshift-service-ca\") pod \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") "
Jul 16 20:32:08.885276 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.885125    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppff6\" (UniqueName: \"kubernetes.io/projected/17a3d412-b301-44e7-b55c-0f2a0805ad9e-kube-api-access-ppff6\") pod \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\" (UID: \"17a3d412-b301-44e7-b55c-0f2a0805ad9e\") "
Jul 16 20:32:08.885489 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.885461    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17a3d412-b301-44e7-b55c-0f2a0805ad9e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "17a3d412-b301-44e7-b55c-0f2a0805ad9e" (UID: "17a3d412-b301-44e7-b55c-0f2a0805ad9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:32:08.887152 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.887130    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17a3d412-b301-44e7-b55c-0f2a0805ad9e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "17a3d412-b301-44e7-b55c-0f2a0805ad9e" (UID: "17a3d412-b301-44e7-b55c-0f2a0805ad9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:32:08.887260 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.887226    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17a3d412-b301-44e7-b55c-0f2a0805ad9e-kube-api-access-ppff6" (OuterVolumeSpecName: "kube-api-access-ppff6") pod "17a3d412-b301-44e7-b55c-0f2a0805ad9e" (UID: "17a3d412-b301-44e7-b55c-0f2a0805ad9e"). InnerVolumeSpecName "kube-api-access-ppff6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:32:08.986018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.985981    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17a3d412-b301-44e7-b55c-0f2a0805ad9e-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:32:08.986018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.986013    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17a3d412-b301-44e7-b55c-0f2a0805ad9e-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:32:08.986018 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:08.986022    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppff6\" (UniqueName: \"kubernetes.io/projected/17a3d412-b301-44e7-b55c-0f2a0805ad9e-kube-api-access-ppff6\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:32:09.059408 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:09.059371    2579 generic.go:356] "Generic (PLEG): container finished" podID="17a3d412-b301-44e7-b55c-0f2a0805ad9e" containerID="ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3" exitCode=0
Jul 16 20:32:09.059590 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:09.059414    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"
Jul 16 20:32:09.059590 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:09.059459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7" event={"ID":"17a3d412-b301-44e7-b55c-0f2a0805ad9e","Type":"ContainerDied","Data":"ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3"}
Jul 16 20:32:09.059590 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:09.059510    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7" event={"ID":"17a3d412-b301-44e7-b55c-0f2a0805ad9e","Type":"ContainerDied","Data":"56fb0f6f8959689e9637b462c4bd3034dd88bfba3d67e8dd1964de82e4b7b7b9"}
Jul 16 20:32:09.059590 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:09.059532    2579 scope.go:117] "RemoveContainer" containerID="ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3"
Jul 16 20:32:09.071356 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:09.071329    2579 scope.go:117] "RemoveContainer" containerID="ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3"
Jul 16 20:32:09.071711 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:32:09.071684    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3\": container with ID starting with ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3 not found: ID does not exist" containerID="ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3"
Jul 16 20:32:09.071821 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:09.071722    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3"} err="failed to get container status \"ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3\": rpc error: code = NotFound desc = could not find container \"ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3\": container with ID starting with ebe92d941f09102fee047121016fb99352441b8a31b3c5fe0c0183072123bec3 not found: ID does not exist"
Jul 16 20:32:09.089279 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:09.089247    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"]
Jul 16 20:32:09.091380 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:09.091347    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9ceb054-7d6594c795-5swz7"]
Jul 16 20:32:10.408374 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:10.408342    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17a3d412-b301-44e7-b55c-0f2a0805ad9e" path="/var/lib/kubelet/pods/17a3d412-b301-44e7-b55c-0f2a0805ad9e/volumes"
Jul 16 20:32:11.094229 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:11.094193    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:32:26.003978 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.003939    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"]
Jul 16 20:32:26.004422 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.004403    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerName="revoke-keys"
Jul 16 20:32:26.004491 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.004424    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerName="revoke-keys"
Jul 16 20:32:26.004491 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.004459    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="249c3330-2d24-4e47-ac22-31a48f8a0ea7" containerName="revoke-keys"
Jul 16 20:32:26.004491 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.004465    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="249c3330-2d24-4e47-ac22-31a48f8a0ea7" containerName="revoke-keys"
Jul 16 20:32:26.004491 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.004475    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17a3d412-b301-44e7-b55c-0f2a0805ad9e" containerName="maas-api"
Jul 16 20:32:26.004491 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.004481    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="17a3d412-b301-44e7-b55c-0f2a0805ad9e" containerName="maas-api"
Jul 16 20:32:26.004643 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.004539    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="249c3330-2d24-4e47-ac22-31a48f8a0ea7" containerName="revoke-keys"
Jul 16 20:32:26.004643 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.004547    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="17a3d412-b301-44e7-b55c-0f2a0805ad9e" containerName="maas-api"
Jul 16 20:32:26.004643 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.004557    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41202d69-db47-4f43-b4c4-1a4c52013c6e" containerName="revoke-keys"
Jul 16 20:32:26.009125 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.009103    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.011656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.011631    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 20:32:26.011798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.011696    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0d61d6f5-serving-cert\""
Jul 16 20:32:26.011798 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.011717    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g78x2\""
Jul 16 20:32:26.012703 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.012573    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 20:32:26.016400 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.016353    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"]
Jul 16 20:32:26.148988 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.148955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxh74\" (UniqueName: \"kubernetes.io/projected/c0803e32-bdbe-463a-8b06-af8da0ca35ac-kube-api-access-kxh74\") pod \"maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.149200 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.149007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0803e32-bdbe-463a-8b06-af8da0ca35ac-openshift-service-ca\") pod \"maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.149200 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.149026    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0803e32-bdbe-463a-8b06-af8da0ca35ac-maas-api-tls\") pod \"maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.250304 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.250260    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxh74\" (UniqueName: \"kubernetes.io/projected/c0803e32-bdbe-463a-8b06-af8da0ca35ac-kube-api-access-kxh74\") pod \"maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.250562 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.250328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0803e32-bdbe-463a-8b06-af8da0ca35ac-openshift-service-ca\") pod \"maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.250562 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.250352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0803e32-bdbe-463a-8b06-af8da0ca35ac-maas-api-tls\") pod \"maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.250562 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:32:26.250491    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-serving-cert: secret "maas-api-e2e-derive-0d61d6f5-serving-cert" not found
Jul 16 20:32:26.250764 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:32:26.250569    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0803e32-bdbe-463a-8b06-af8da0ca35ac-maas-api-tls podName:c0803e32-bdbe-463a-8b06-af8da0ca35ac nodeName:}" failed. No retries permitted until 2026-07-16 20:32:26.750548229 +0000 UTC m=+1910.898643631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0803e32-bdbe-463a-8b06-af8da0ca35ac-maas-api-tls") pod "maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h" (UID: "c0803e32-bdbe-463a-8b06-af8da0ca35ac") : secret "maas-api-e2e-derive-0d61d6f5-serving-cert" not found
Jul 16 20:32:26.251048 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.251023    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0803e32-bdbe-463a-8b06-af8da0ca35ac-openshift-service-ca\") pod \"maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.260661 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.260601    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxh74\" (UniqueName: \"kubernetes.io/projected/c0803e32-bdbe-463a-8b06-af8da0ca35ac-kube-api-access-kxh74\") pod \"maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.755284 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.755243    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0803e32-bdbe-463a-8b06-af8da0ca35ac-maas-api-tls\") pod \"maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.757723 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.757689    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0803e32-bdbe-463a-8b06-af8da0ca35ac-maas-api-tls\") pod \"maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:26.922638 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:26.922598    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:27.054713 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:27.054629    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"]
Jul 16 20:32:27.057576 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:32:27.057547    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0803e32_bdbe_463a_8b06_af8da0ca35ac.slice/crio-e35ba4dc058e280798a8f1c7e2cb51de6c1aee1704c7361b4a7084f61877b92a WatchSource:0}: Error finding container e35ba4dc058e280798a8f1c7e2cb51de6c1aee1704c7361b4a7084f61877b92a: Status 404 returned error can't find the container with id e35ba4dc058e280798a8f1c7e2cb51de6c1aee1704c7361b4a7084f61877b92a
Jul 16 20:32:27.160762 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:27.160726    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h" event={"ID":"c0803e32-bdbe-463a-8b06-af8da0ca35ac","Type":"ContainerStarted","Data":"e35ba4dc058e280798a8f1c7e2cb51de6c1aee1704c7361b4a7084f61877b92a"}
Jul 16 20:32:28.171470 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:28.171421    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h" event={"ID":"c0803e32-bdbe-463a-8b06-af8da0ca35ac","Type":"ContainerStarted","Data":"d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44"}
Jul 16 20:32:28.171873 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:28.171545    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:28.188811 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:28.188756    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h" podStartSLOduration=2.853088317 podStartE2EDuration="3.18873942s" totalImagesPullingTime="335.651103ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:28.187516386 +0000 UTC m=+1912.335611905" watchObservedRunningTime="2026-07-16 20:32:28.18873942 +0000 UTC m=+1912.336834840"
Jul 16 20:32:30.187365 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:30.187333    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"]
Jul 16 20:32:30.190952 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:30.190933    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"
Jul 16 20:32:30.193066 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:30.193040    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:32:30.200067 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:30.200036    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"]
Jul 16 20:32:30.287610 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:30.287572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f494ca-cfc3-483b-8fcd-c7e75dd77ede-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km\" (UID: \"58f494ca-cfc3-483b-8fcd-c7e75dd77ede\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"
Jul 16 20:32:30.388138 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:30.388086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f494ca-cfc3-483b-8fcd-c7e75dd77ede-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km\" (UID: \"58f494ca-cfc3-483b-8fcd-c7e75dd77ede\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"
Jul 16 20:32:30.388809 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:30.388786    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f494ca-cfc3-483b-8fcd-c7e75dd77ede-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km\" (UID: \"58f494ca-cfc3-483b-8fcd-c7e75dd77ede\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"
Jul 16 20:32:30.503681 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:30.503595    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"
Jul 16 20:32:30.639308 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:30.639274    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"]
Jul 16 20:32:30.642704 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:32:30.642674    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58f494ca_cfc3_483b_8fcd_c7e75dd77ede.slice/crio-58ca921978317ff3ebd76e3f59b6e2273c175ed5b4a3170712afb1a071295fea WatchSource:0}: Error finding container 58ca921978317ff3ebd76e3f59b6e2273c175ed5b4a3170712afb1a071295fea: Status 404 returned error can't find the container with id 58ca921978317ff3ebd76e3f59b6e2273c175ed5b4a3170712afb1a071295fea
Jul 16 20:32:31.191823 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:31.191786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km" event={"ID":"58f494ca-cfc3-483b-8fcd-c7e75dd77ede","Type":"ContainerStarted","Data":"7deb881282c6b3d836e9550a76a1525047551788a5e8bae5486636fb7276645e"}
Jul 16 20:32:31.192228 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:31.191849    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km" event={"ID":"58f494ca-cfc3-483b-8fcd-c7e75dd77ede","Type":"ContainerStarted","Data":"58ca921978317ff3ebd76e3f59b6e2273c175ed5b4a3170712afb1a071295fea"}
Jul 16 20:32:31.227597 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:31.227549    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km" podStartSLOduration=1.227531846 podStartE2EDuration="1.227531846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:31.225864892 +0000 UTC m=+1915.373960324" watchObservedRunningTime="2026-07-16 20:32:31.227531846 +0000 UTC m=+1915.375627267"
Jul 16 20:32:32.199082 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:32.199044    2579 generic.go:356] "Generic (PLEG): container finished" podID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerID="7deb881282c6b3d836e9550a76a1525047551788a5e8bae5486636fb7276645e" exitCode=7
Jul 16 20:32:32.199484 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:32.199125    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km" event={"ID":"58f494ca-cfc3-483b-8fcd-c7e75dd77ede","Type":"ContainerDied","Data":"7deb881282c6b3d836e9550a76a1525047551788a5e8bae5486636fb7276645e"}
Jul 16 20:32:32.199484 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:32.199329    2579 scope.go:117] "RemoveContainer" containerID="7deb881282c6b3d836e9550a76a1525047551788a5e8bae5486636fb7276645e"
Jul 16 20:32:33.206559 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:33.206523    2579 generic.go:356] "Generic (PLEG): container finished" podID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerID="cf72358af5bb8606589b38ddce19bfabb5d343ff21931c65b6dd63126e843fb7" exitCode=7
Jul 16 20:32:33.207025 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:33.206608    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km" event={"ID":"58f494ca-cfc3-483b-8fcd-c7e75dd77ede","Type":"ContainerDied","Data":"cf72358af5bb8606589b38ddce19bfabb5d343ff21931c65b6dd63126e843fb7"}
Jul 16 20:32:33.207025 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:33.206670    2579 scope.go:117] "RemoveContainer" containerID="7deb881282c6b3d836e9550a76a1525047551788a5e8bae5486636fb7276645e"
Jul 16 20:32:33.207025 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:33.206885    2579 scope.go:117] "RemoveContainer" containerID="cf72358af5bb8606589b38ddce19bfabb5d343ff21931c65b6dd63126e843fb7"
Jul 16 20:32:33.207197 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:32:33.207141    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km_odh-ai-gateway-infra(58f494ca-cfc3-483b-8fcd-c7e75dd77ede)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede"
Jul 16 20:32:33.486627 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:33.486537    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:32:34.184226 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:34.184198    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:32:34.215129 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:34.215098    2579 scope.go:117] "RemoveContainer" containerID="cf72358af5bb8606589b38ddce19bfabb5d343ff21931c65b6dd63126e843fb7"
Jul 16 20:32:34.215556 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:32:34.215347    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km_odh-ai-gateway-infra(58f494ca-cfc3-483b-8fcd-c7e75dd77ede)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede"
Jul 16 20:32:45.398793 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:45.398756    2579 scope.go:117] "RemoveContainer" containerID="cf72358af5bb8606589b38ddce19bfabb5d343ff21931c65b6dd63126e843fb7"
Jul 16 20:32:46.285398 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:46.285367    2579 generic.go:356] "Generic (PLEG): container finished" podID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerID="96f876608435a841846badff9bb5faf31e3c421afdcef69bcae80786144225c5" exitCode=0
Jul 16 20:32:46.285594 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:46.285467    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km" event={"ID":"58f494ca-cfc3-483b-8fcd-c7e75dd77ede","Type":"ContainerDied","Data":"96f876608435a841846badff9bb5faf31e3c421afdcef69bcae80786144225c5"}
Jul 16 20:32:46.285594 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:46.285535    2579 scope.go:117] "RemoveContainer" containerID="cf72358af5bb8606589b38ddce19bfabb5d343ff21931c65b6dd63126e843fb7"
Jul 16 20:32:47.320740 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:47.320704    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"]
Jul 16 20:32:47.423159 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:47.423132    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"
Jul 16 20:32:47.532942 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:47.532902    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f494ca-cfc3-483b-8fcd-c7e75dd77ede-maas-api-service-ca\") pod \"58f494ca-cfc3-483b-8fcd-c7e75dd77ede\" (UID: \"58f494ca-cfc3-483b-8fcd-c7e75dd77ede\") "
Jul 16 20:32:47.533276 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:47.533253    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58f494ca-cfc3-483b-8fcd-c7e75dd77ede-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "58f494ca-cfc3-483b-8fcd-c7e75dd77ede" (UID: "58f494ca-cfc3-483b-8fcd-c7e75dd77ede"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:32:47.634265 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:47.634174    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f494ca-cfc3-483b-8fcd-c7e75dd77ede-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:32:48.299613 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:48.299581    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"
Jul 16 20:32:48.299789 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:48.299577    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km" event={"ID":"58f494ca-cfc3-483b-8fcd-c7e75dd77ede","Type":"ContainerDied","Data":"58ca921978317ff3ebd76e3f59b6e2273c175ed5b4a3170712afb1a071295fea"}
Jul 16 20:32:48.299789 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:48.299710    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58ca921978317ff3ebd76e3f59b6e2273c175ed5b4a3170712afb1a071295fea"
Jul 16 20:32:48.329792 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:48.329764    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"]
Jul 16 20:32:48.333249 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:48.333221    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-2c4km"]
Jul 16 20:32:48.404482 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:32:48.404424    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" path="/var/lib/kubelet/pods/58f494ca-cfc3-483b-8fcd-c7e75dd77ede/volumes"
Jul 16 20:33:20.231219 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.231181    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89"]
Jul 16 20:33:20.231696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.231579    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerName="revoke-keys"
Jul 16 20:33:20.231696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.231593    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerName="revoke-keys"
Jul 16 20:33:20.231696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.231620    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerName="revoke-keys"
Jul 16 20:33:20.231696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.231626    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerName="revoke-keys"
Jul 16 20:33:20.231696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.231635    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerName="revoke-keys"
Jul 16 20:33:20.231696 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.231641    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerName="revoke-keys"
Jul 16 20:33:20.231885 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.231705    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerName="revoke-keys"
Jul 16 20:33:20.231885 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.231713    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerName="revoke-keys"
Jul 16 20:33:20.231885 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.231720    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="58f494ca-cfc3-483b-8fcd-c7e75dd77ede" containerName="revoke-keys"
Jul 16 20:33:20.234799 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.234781    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89"
Jul 16 20:33:20.237952 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.237928    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:33:20.244624 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.244597    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89"]
Jul 16 20:33:20.339230 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.339198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1e3f8d-b7aa-465e-ad72-021624cd6cd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89\" (UID: \"fd1e3f8d-b7aa-465e-ad72-021624cd6cd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89"
Jul 16 20:33:20.440287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.440239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1e3f8d-b7aa-465e-ad72-021624cd6cd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89\" (UID: \"fd1e3f8d-b7aa-465e-ad72-021624cd6cd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89"
Jul 16 20:33:20.440911 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.440888    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1e3f8d-b7aa-465e-ad72-021624cd6cd5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89\" (UID: \"fd1e3f8d-b7aa-465e-ad72-021624cd6cd5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89"
Jul 16 20:33:20.546690 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.546587    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89"
Jul 16 20:33:20.680480 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:20.680453    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89"]
Jul 16 20:33:20.682227 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:33:20.682196    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1e3f8d_b7aa_465e_ad72_021624cd6cd5.slice/crio-e5868bbef3b353ead7adbd5ebfadb6b48a2dcc9f883e47e55d59273c6024e9db WatchSource:0}: Error finding container e5868bbef3b353ead7adbd5ebfadb6b48a2dcc9f883e47e55d59273c6024e9db: Status 404 returned error can't find the container with id e5868bbef3b353ead7adbd5ebfadb6b48a2dcc9f883e47e55d59273c6024e9db
Jul 16 20:33:21.481545 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:21.481511    2579 generic.go:356] "Generic (PLEG): container finished" podID="fd1e3f8d-b7aa-465e-ad72-021624cd6cd5" containerID="242f22202da2a3a5c7716dc5fc83ee1a82ea2a8c33c20ac406c41666dd29904f" exitCode=0
Jul 16 20:33:21.481948 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:21.481587    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89" event={"ID":"fd1e3f8d-b7aa-465e-ad72-021624cd6cd5","Type":"ContainerDied","Data":"242f22202da2a3a5c7716dc5fc83ee1a82ea2a8c33c20ac406c41666dd29904f"}
Jul 16 20:33:21.481948 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:21.481628    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89" event={"ID":"fd1e3f8d-b7aa-465e-ad72-021624cd6cd5","Type":"ContainerStarted","Data":"e5868bbef3b353ead7adbd5ebfadb6b48a2dcc9f883e47e55d59273c6024e9db"}
Jul 16 20:33:22.620286 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:22.620259    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89"
Jul 16 20:33:22.661501 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:22.661467    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1e3f8d-b7aa-465e-ad72-021624cd6cd5-maas-api-service-ca\") pod \"fd1e3f8d-b7aa-465e-ad72-021624cd6cd5\" (UID: \"fd1e3f8d-b7aa-465e-ad72-021624cd6cd5\") "
Jul 16 20:33:22.661872 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:22.661852    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd1e3f8d-b7aa-465e-ad72-021624cd6cd5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fd1e3f8d-b7aa-465e-ad72-021624cd6cd5" (UID: "fd1e3f8d-b7aa-465e-ad72-021624cd6cd5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:33:22.762848 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:22.762765    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1e3f8d-b7aa-465e-ad72-021624cd6cd5-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:33:23.494276 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:23.494238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89" event={"ID":"fd1e3f8d-b7aa-465e-ad72-021624cd6cd5","Type":"ContainerDied","Data":"e5868bbef3b353ead7adbd5ebfadb6b48a2dcc9f883e47e55d59273c6024e9db"}
Jul 16 20:33:23.494517 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:23.494289    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5868bbef3b353ead7adbd5ebfadb6b48a2dcc9f883e47e55d59273c6024e9db"
Jul 16 20:33:23.494517 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:23.494250    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89"
Jul 16 20:33:30.863664 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:30.863575    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"]
Jul 16 20:33:30.864113 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:30.863849    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h" podUID="c0803e32-bdbe-463a-8b06-af8da0ca35ac" containerName="maas-api" containerID="cri-o://d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44" gracePeriod=30
Jul 16 20:33:31.106658 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.106633    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:33:31.242028 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.241984    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0803e32-bdbe-463a-8b06-af8da0ca35ac-maas-api-tls\") pod \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") "
Jul 16 20:33:31.242218 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.242052    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0803e32-bdbe-463a-8b06-af8da0ca35ac-openshift-service-ca\") pod \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") "
Jul 16 20:33:31.242218 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.242111    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxh74\" (UniqueName: \"kubernetes.io/projected/c0803e32-bdbe-463a-8b06-af8da0ca35ac-kube-api-access-kxh74\") pod \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\" (UID: \"c0803e32-bdbe-463a-8b06-af8da0ca35ac\") "
Jul 16 20:33:31.242451 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.242404    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0803e32-bdbe-463a-8b06-af8da0ca35ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0803e32-bdbe-463a-8b06-af8da0ca35ac" (UID: "c0803e32-bdbe-463a-8b06-af8da0ca35ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:33:31.244166 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.244142    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0803e32-bdbe-463a-8b06-af8da0ca35ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0803e32-bdbe-463a-8b06-af8da0ca35ac" (UID: "c0803e32-bdbe-463a-8b06-af8da0ca35ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:33:31.244297 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.244277    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0803e32-bdbe-463a-8b06-af8da0ca35ac-kube-api-access-kxh74" (OuterVolumeSpecName: "kube-api-access-kxh74") pod "c0803e32-bdbe-463a-8b06-af8da0ca35ac" (UID: "c0803e32-bdbe-463a-8b06-af8da0ca35ac"). InnerVolumeSpecName "kube-api-access-kxh74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:33:31.343588 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.343551    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxh74\" (UniqueName: \"kubernetes.io/projected/c0803e32-bdbe-463a-8b06-af8da0ca35ac-kube-api-access-kxh74\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:33:31.343588 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.343585    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0803e32-bdbe-463a-8b06-af8da0ca35ac-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:33:31.343588 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.343595    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0803e32-bdbe-463a-8b06-af8da0ca35ac-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:33:31.543464 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.543342    2579 generic.go:356] "Generic (PLEG): container finished" podID="c0803e32-bdbe-463a-8b06-af8da0ca35ac" containerID="d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44" exitCode=0
Jul 16 20:33:31.543464 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.543391    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"
Jul 16 20:33:31.543464 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.543427    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h" event={"ID":"c0803e32-bdbe-463a-8b06-af8da0ca35ac","Type":"ContainerDied","Data":"d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44"}
Jul 16 20:33:31.543763 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.543506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h" event={"ID":"c0803e32-bdbe-463a-8b06-af8da0ca35ac","Type":"ContainerDied","Data":"e35ba4dc058e280798a8f1c7e2cb51de6c1aee1704c7361b4a7084f61877b92a"}
Jul 16 20:33:31.543763 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.543529    2579 scope.go:117] "RemoveContainer" containerID="d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44"
Jul 16 20:33:31.552845 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.552827    2579 scope.go:117] "RemoveContainer" containerID="d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44"
Jul 16 20:33:31.553127 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:33:31.553105    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44\": container with ID starting with d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44 not found: ID does not exist" containerID="d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44"
Jul 16 20:33:31.553173 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.553136    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44"} err="failed to get container status \"d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44\": rpc error: code = NotFound desc = could not find container \"d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44\": container with ID starting with d26166f66482a4dd6f6522bf5b55520e206f3a68d0bdf07bd12ec748240e5f44 not found: ID does not exist"
Jul 16 20:33:31.565855 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.565823    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"]
Jul 16 20:33:31.568056 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:31.568034    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d61d6f5-7ddff8fdc7-8q48h"]
Jul 16 20:33:32.404591 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:32.404554    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0803e32-bdbe-463a-8b06-af8da0ca35ac" path="/var/lib/kubelet/pods/c0803e32-bdbe-463a-8b06-af8da0ca35ac/volumes"
Jul 16 20:33:33.987664 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:33:33.987622    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:35:08.287040 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:08.286996    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:35:17.881620 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:17.881580    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:35:24.233611 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.233573    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"]
Jul 16 20:35:24.234209 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.234192    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd1e3f8d-b7aa-465e-ad72-021624cd6cd5" containerName="revoke-keys"
Jul 16 20:35:24.234254 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.234213    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd1e3f8d-b7aa-465e-ad72-021624cd6cd5" containerName="revoke-keys"
Jul 16 20:35:24.234254 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.234231    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0803e32-bdbe-463a-8b06-af8da0ca35ac" containerName="maas-api"
Jul 16 20:35:24.234254 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.234240    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0803e32-bdbe-463a-8b06-af8da0ca35ac" containerName="maas-api"
Jul 16 20:35:24.234366 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.234326    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd1e3f8d-b7aa-465e-ad72-021624cd6cd5" containerName="revoke-keys"
Jul 16 20:35:24.234366 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.234338    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0803e32-bdbe-463a-8b06-af8da0ca35ac" containerName="maas-api"
Jul 16 20:35:24.237844 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.237826    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.240479 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.240456    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 20:35:24.240916 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.240894    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g78x2\""
Jul 16 20:35:24.241181 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.241167    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 20:35:24.241595 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.241551    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-087d0429-serving-cert\""
Jul 16 20:35:24.247671 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.247643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"]
Jul 16 20:35:24.365823 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.365788    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-maas-api-tls\") pod \"maas-api-e2e-mt-087d0429-5666bf46f5-slr82\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.366045 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.365840    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-openshift-service-ca\") pod \"maas-api-e2e-mt-087d0429-5666bf46f5-slr82\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.366045 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.365987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6zp2\" (UniqueName: \"kubernetes.io/projected/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-kube-api-access-w6zp2\") pod \"maas-api-e2e-mt-087d0429-5666bf46f5-slr82\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.466773 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.466730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-maas-api-tls\") pod \"maas-api-e2e-mt-087d0429-5666bf46f5-slr82\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.466994 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.466799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-openshift-service-ca\") pod \"maas-api-e2e-mt-087d0429-5666bf46f5-slr82\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.466994 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.466915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6zp2\" (UniqueName: \"kubernetes.io/projected/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-kube-api-access-w6zp2\") pod \"maas-api-e2e-mt-087d0429-5666bf46f5-slr82\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.467554 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.467529    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-openshift-service-ca\") pod \"maas-api-e2e-mt-087d0429-5666bf46f5-slr82\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.469312 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.469293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-maas-api-tls\") pod \"maas-api-e2e-mt-087d0429-5666bf46f5-slr82\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.478244 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.478218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6zp2\" (UniqueName: \"kubernetes.io/projected/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-kube-api-access-w6zp2\") pod \"maas-api-e2e-mt-087d0429-5666bf46f5-slr82\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.550786 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.550697    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:24.699483 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.699456    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"]
Jul 16 20:35:24.701313 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:35:24.701288    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb790ae1_eb18_4a30_8db8_7fb41608d9b6.slice/crio-c8699ddcce560d7ed83f8ca7d56b23d5a48a5d3c41644799414fe8e2ff7be0be WatchSource:0}: Error finding container c8699ddcce560d7ed83f8ca7d56b23d5a48a5d3c41644799414fe8e2ff7be0be: Status 404 returned error can't find the container with id c8699ddcce560d7ed83f8ca7d56b23d5a48a5d3c41644799414fe8e2ff7be0be
Jul 16 20:35:24.703116 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:24.703095    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 20:35:25.204884 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:25.204844    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82" event={"ID":"bb790ae1-eb18-4a30-8db8-7fb41608d9b6","Type":"ContainerStarted","Data":"361d79649f239fc8933f8a9b2ad62a63e3bfc4e13658706d6898be94ff7259ce"}
Jul 16 20:35:25.205064 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:25.204911    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82" event={"ID":"bb790ae1-eb18-4a30-8db8-7fb41608d9b6","Type":"ContainerStarted","Data":"c8699ddcce560d7ed83f8ca7d56b23d5a48a5d3c41644799414fe8e2ff7be0be"}
Jul 16 20:35:25.205064 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:25.204960    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:25.221878 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:25.221820    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82" podStartSLOduration=0.81324115 podStartE2EDuration="1.221776421s" totalImagesPullingTime="408.535271ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:35:25.221220314 +0000 UTC m=+2089.369315738" watchObservedRunningTime="2026-07-16 20:35:25.221776421 +0000 UTC m=+2089.369871844"
Jul 16 20:35:28.686369 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:28.686331    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:35:31.215741 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:31.215712    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:35.762200 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:35.762149    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl"]
Jul 16 20:35:35.766063 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:35.766045    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl"
Jul 16 20:35:35.769100 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:35.769071    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:35:35.777892 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:35.777862    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl"]
Jul 16 20:35:35.874284 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:35.874246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29186dd-9ad3-474f-b4e8-763027b7e7ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-087d0429-r48nl\" (UID: \"d29186dd-9ad3-474f-b4e8-763027b7e7ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl"
Jul 16 20:35:35.975334 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:35.975296    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29186dd-9ad3-474f-b4e8-763027b7e7ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-087d0429-r48nl\" (UID: \"d29186dd-9ad3-474f-b4e8-763027b7e7ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl"
Jul 16 20:35:35.975983 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:35.975960    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29186dd-9ad3-474f-b4e8-763027b7e7ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-087d0429-r48nl\" (UID: \"d29186dd-9ad3-474f-b4e8-763027b7e7ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl"
Jul 16 20:35:36.078555 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:36.078467    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl"
Jul 16 20:35:36.218932 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:36.218902    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl"]
Jul 16 20:35:36.220665 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:35:36.220639    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29186dd_9ad3_474f_b4e8_763027b7e7ee.slice/crio-9f90dfa63f00e286837c7b64d5d77c4881080ddb34a88bf3e3580afae37d67e8 WatchSource:0}: Error finding container 9f90dfa63f00e286837c7b64d5d77c4881080ddb34a88bf3e3580afae37d67e8: Status 404 returned error can't find the container with id 9f90dfa63f00e286837c7b64d5d77c4881080ddb34a88bf3e3580afae37d67e8
Jul 16 20:35:36.269936 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:36.269902    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl" event={"ID":"d29186dd-9ad3-474f-b4e8-763027b7e7ee","Type":"ContainerStarted","Data":"9f90dfa63f00e286837c7b64d5d77c4881080ddb34a88bf3e3580afae37d67e8"}
Jul 16 20:35:37.277044 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:37.276946    2579 generic.go:356] "Generic (PLEG): container finished" podID="d29186dd-9ad3-474f-b4e8-763027b7e7ee" containerID="10a5d9e53b0795455d80a4bf3142567d19d9bce1760f0860837cd5920bdcbbbc" exitCode=0
Jul 16 20:35:37.277044 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:37.277035    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl" event={"ID":"d29186dd-9ad3-474f-b4e8-763027b7e7ee","Type":"ContainerDied","Data":"10a5d9e53b0795455d80a4bf3142567d19d9bce1760f0860837cd5920bdcbbbc"}
Jul 16 20:35:38.426326 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:38.426292    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl"
Jul 16 20:35:38.600577 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:38.600482    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29186dd-9ad3-474f-b4e8-763027b7e7ee-maas-api-service-ca\") pod \"d29186dd-9ad3-474f-b4e8-763027b7e7ee\" (UID: \"d29186dd-9ad3-474f-b4e8-763027b7e7ee\") "
Jul 16 20:35:38.600850 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:38.600826    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d29186dd-9ad3-474f-b4e8-763027b7e7ee-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d29186dd-9ad3-474f-b4e8-763027b7e7ee" (UID: "d29186dd-9ad3-474f-b4e8-763027b7e7ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:35:38.702062 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:38.702015    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d29186dd-9ad3-474f-b4e8-763027b7e7ee-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:35:39.290402 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:39.290373    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl"
Jul 16 20:35:39.290608 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:39.290409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-087d0429-r48nl" event={"ID":"d29186dd-9ad3-474f-b4e8-763027b7e7ee","Type":"ContainerDied","Data":"9f90dfa63f00e286837c7b64d5d77c4881080ddb34a88bf3e3580afae37d67e8"}
Jul 16 20:35:39.290608 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:39.290473    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f90dfa63f00e286837c7b64d5d77c4881080ddb34a88bf3e3580afae37d67e8"
Jul 16 20:35:46.186815 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.186779    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"]
Jul 16 20:35:46.187350 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.187046    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82" podUID="bb790ae1-eb18-4a30-8db8-7fb41608d9b6" containerName="maas-api" containerID="cri-o://361d79649f239fc8933f8a9b2ad62a63e3bfc4e13658706d6898be94ff7259ce" gracePeriod=30
Jul 16 20:35:46.211318 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.211270    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82" podUID="bb790ae1-eb18-4a30-8db8-7fb41608d9b6" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.52:8443/health\": dial tcp 10.134.0.52:8443: connect: connection refused"
Jul 16 20:35:46.331885 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.331851    2579 generic.go:356] "Generic (PLEG): container finished" podID="bb790ae1-eb18-4a30-8db8-7fb41608d9b6" containerID="361d79649f239fc8933f8a9b2ad62a63e3bfc4e13658706d6898be94ff7259ce" exitCode=0
Jul 16 20:35:46.332068 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.331921    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82" event={"ID":"bb790ae1-eb18-4a30-8db8-7fb41608d9b6","Type":"ContainerDied","Data":"361d79649f239fc8933f8a9b2ad62a63e3bfc4e13658706d6898be94ff7259ce"}
Jul 16 20:35:46.440650 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.440586    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:46.575480 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.575420    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-maas-api-tls\") pod \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") "
Jul 16 20:35:46.575689 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.575531    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6zp2\" (UniqueName: \"kubernetes.io/projected/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-kube-api-access-w6zp2\") pod \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") "
Jul 16 20:35:46.575689 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.575561    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-openshift-service-ca\") pod \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\" (UID: \"bb790ae1-eb18-4a30-8db8-7fb41608d9b6\") "
Jul 16 20:35:46.575929 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.575905    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb790ae1-eb18-4a30-8db8-7fb41608d9b6" (UID: "bb790ae1-eb18-4a30-8db8-7fb41608d9b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:35:46.577709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.577674    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-kube-api-access-w6zp2" (OuterVolumeSpecName: "kube-api-access-w6zp2") pod "bb790ae1-eb18-4a30-8db8-7fb41608d9b6" (UID: "bb790ae1-eb18-4a30-8db8-7fb41608d9b6"). InnerVolumeSpecName "kube-api-access-w6zp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:35:46.577709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.577694    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb790ae1-eb18-4a30-8db8-7fb41608d9b6" (UID: "bb790ae1-eb18-4a30-8db8-7fb41608d9b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:35:46.676360 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.676320    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:35:46.676360 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.676355    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:35:46.676360 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:46.676364    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6zp2\" (UniqueName: \"kubernetes.io/projected/bb790ae1-eb18-4a30-8db8-7fb41608d9b6-kube-api-access-w6zp2\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:35:47.339176 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:47.339141    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"
Jul 16 20:35:47.339656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:47.339133    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82" event={"ID":"bb790ae1-eb18-4a30-8db8-7fb41608d9b6","Type":"ContainerDied","Data":"c8699ddcce560d7ed83f8ca7d56b23d5a48a5d3c41644799414fe8e2ff7be0be"}
Jul 16 20:35:47.339656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:47.339280    2579 scope.go:117] "RemoveContainer" containerID="361d79649f239fc8933f8a9b2ad62a63e3bfc4e13658706d6898be94ff7259ce"
Jul 16 20:35:47.363792 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:47.363758    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"]
Jul 16 20:35:47.370653 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:47.370625    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-087d0429-5666bf46f5-slr82"]
Jul 16 20:35:48.404499 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:48.404455    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb790ae1-eb18-4a30-8db8-7fb41608d9b6" path="/var/lib/kubelet/pods/bb790ae1-eb18-4a30-8db8-7fb41608d9b6/volumes"
Jul 16 20:35:49.397752 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:35:49.397713    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:36:24.578283 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.578232    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"]
Jul 16 20:36:24.581017 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.578856    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d29186dd-9ad3-474f-b4e8-763027b7e7ee" containerName="revoke-keys"
Jul 16 20:36:24.581017 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.578878    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d29186dd-9ad3-474f-b4e8-763027b7e7ee" containerName="revoke-keys"
Jul 16 20:36:24.581017 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.578928    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb790ae1-eb18-4a30-8db8-7fb41608d9b6" containerName="maas-api"
Jul 16 20:36:24.581017 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.578936    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb790ae1-eb18-4a30-8db8-7fb41608d9b6" containerName="maas-api"
Jul 16 20:36:24.581017 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.579049    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb790ae1-eb18-4a30-8db8-7fb41608d9b6" containerName="maas-api"
Jul 16 20:36:24.581017 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.579063    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d29186dd-9ad3-474f-b4e8-763027b7e7ee" containerName="revoke-keys"
Jul 16 20:36:24.582547 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.582523    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:24.584890 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.584868    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0faadf3f-serving-cert\""
Jul 16 20:36:24.585038 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.584895    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 20:36:24.586188 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.586162    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 20:36:24.586390 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.586257    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g78x2\""
Jul 16 20:36:24.589858 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.589783    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"]
Jul 16 20:36:24.741828 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.741783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce97e411-6495-40e3-a368-69907848cdd0-openshift-service-ca\") pod \"maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:24.742078 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.741941    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97e411-6495-40e3-a368-69907848cdd0-maas-api-tls\") pod \"maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:24.742078 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.741995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8rb2\" (UniqueName: \"kubernetes.io/projected/ce97e411-6495-40e3-a368-69907848cdd0-kube-api-access-d8rb2\") pod \"maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:24.843500 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.843361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97e411-6495-40e3-a368-69907848cdd0-maas-api-tls\") pod \"maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:24.843500 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.843420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8rb2\" (UniqueName: \"kubernetes.io/projected/ce97e411-6495-40e3-a368-69907848cdd0-kube-api-access-d8rb2\") pod \"maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:24.843776 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:36:24.843533    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-serving-cert: secret "maas-api-e2e-mt-0faadf3f-serving-cert" not found
Jul 16 20:36:24.843776 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.843546    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce97e411-6495-40e3-a368-69907848cdd0-openshift-service-ca\") pod \"maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:24.843776 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:36:24.843610    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce97e411-6495-40e3-a368-69907848cdd0-maas-api-tls podName:ce97e411-6495-40e3-a368-69907848cdd0 nodeName:}" failed. No retries permitted until 2026-07-16 20:36:25.343591476 +0000 UTC m=+2149.491686880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce97e411-6495-40e3-a368-69907848cdd0-maas-api-tls") pod "maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd" (UID: "ce97e411-6495-40e3-a368-69907848cdd0") : secret "maas-api-e2e-mt-0faadf3f-serving-cert" not found
Jul 16 20:36:24.844265 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.844240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce97e411-6495-40e3-a368-69907848cdd0-openshift-service-ca\") pod \"maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:24.859171 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:24.859136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8rb2\" (UniqueName: \"kubernetes.io/projected/ce97e411-6495-40e3-a368-69907848cdd0-kube-api-access-d8rb2\") pod \"maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:25.347836 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:25.347798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97e411-6495-40e3-a368-69907848cdd0-maas-api-tls\") pod \"maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:25.350394 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:25.350371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97e411-6495-40e3-a368-69907848cdd0-maas-api-tls\") pod \"maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:25.496187 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:25.496139    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:25.643664 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:25.643635    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"]
Jul 16 20:36:25.645662 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:36:25.645633    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce97e411_6495_40e3_a368_69907848cdd0.slice/crio-6441a99151dd82b17075181c3698058b3065b9798b4569a2333f4ff9717b892b WatchSource:0}: Error finding container 6441a99151dd82b17075181c3698058b3065b9798b4569a2333f4ff9717b892b: Status 404 returned error can't find the container with id 6441a99151dd82b17075181c3698058b3065b9798b4569a2333f4ff9717b892b
Jul 16 20:36:26.569132 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:26.569094    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd" event={"ID":"ce97e411-6495-40e3-a368-69907848cdd0","Type":"ContainerStarted","Data":"6441a99151dd82b17075181c3698058b3065b9798b4569a2333f4ff9717b892b"}
Jul 16 20:36:27.383902 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:27.383805    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:36:27.577087 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:27.577045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd" event={"ID":"ce97e411-6495-40e3-a368-69907848cdd0","Type":"ContainerStarted","Data":"fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63"}
Jul 16 20:36:27.577251 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:27.577125    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:27.594561 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:27.594512    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd" podStartSLOduration=2.151770361 podStartE2EDuration="3.594495899s" totalImagesPullingTime="1.442725538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:36:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:36:27.593941253 +0000 UTC m=+2151.742036677" watchObservedRunningTime="2026-07-16 20:36:27.594495899 +0000 UTC m=+2151.742591322"
Jul 16 20:36:33.589286 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:33.589255    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:36:36.512862 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:36.512822    2579 scope.go:117] "RemoveContainer" containerID="8563ce99aa230264533b6178f151a247b394d5c233e441a93d1ecae213f940d7"
Jul 16 20:36:36.521930 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:36.521907    2579 scope.go:117] "RemoveContainer" containerID="0e5d087f1a3e61b3e5f623a427ab058d8aeac2e6050eadf916eee7aedbdc18ab"
Jul 16 20:36:38.179008 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:38.178965    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:36:56.190062 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:36:56.190026    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:37:01.809419 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:01.809376    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb"]
Jul 16 20:37:01.813336 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:01.813314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb"
Jul 16 20:37:01.815587 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:01.815564    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:37:01.820475 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:01.820446    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb"]
Jul 16 20:37:01.870307 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:01.870267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a260ab4-4a87-491e-81ad-893e08424d43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb\" (UID: \"6a260ab4-4a87-491e-81ad-893e08424d43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb"
Jul 16 20:37:01.971746 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:01.971709    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a260ab4-4a87-491e-81ad-893e08424d43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb\" (UID: \"6a260ab4-4a87-491e-81ad-893e08424d43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb"
Jul 16 20:37:01.972388 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:01.972367    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a260ab4-4a87-491e-81ad-893e08424d43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb\" (UID: \"6a260ab4-4a87-491e-81ad-893e08424d43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb"
Jul 16 20:37:02.126149 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:02.126058    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb"
Jul 16 20:37:02.273788 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:02.273762    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb"]
Jul 16 20:37:02.275514 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:37:02.275481    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a260ab4_4a87_491e_81ad_893e08424d43.slice/crio-5645b41a497906730702bd753d2c91992a838597ae1237090b8835cfb56293e2 WatchSource:0}: Error finding container 5645b41a497906730702bd753d2c91992a838597ae1237090b8835cfb56293e2: Status 404 returned error can't find the container with id 5645b41a497906730702bd753d2c91992a838597ae1237090b8835cfb56293e2
Jul 16 20:37:02.787084 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:02.786988    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb" event={"ID":"6a260ab4-4a87-491e-81ad-893e08424d43","Type":"ContainerStarted","Data":"ac8eac4c10501c2225b337f8910b56b1dc2e2d782270153faf8c218a9ef12094"}
Jul 16 20:37:02.787084 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:02.787043    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb" event={"ID":"6a260ab4-4a87-491e-81ad-893e08424d43","Type":"ContainerStarted","Data":"5645b41a497906730702bd753d2c91992a838597ae1237090b8835cfb56293e2"}
Jul 16 20:37:02.803374 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:02.803321    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb" podStartSLOduration=1.80330349 podStartE2EDuration="1.80330349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:37:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:37:02.802260162 +0000 UTC m=+2186.950355586" watchObservedRunningTime="2026-07-16 20:37:02.80330349 +0000 UTC m=+2186.951398913"
Jul 16 20:37:03.794236 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:03.794199    2579 generic.go:356] "Generic (PLEG): container finished" podID="6a260ab4-4a87-491e-81ad-893e08424d43" containerID="ac8eac4c10501c2225b337f8910b56b1dc2e2d782270153faf8c218a9ef12094" exitCode=0
Jul 16 20:37:03.794647 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:03.794281    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb" event={"ID":"6a260ab4-4a87-491e-81ad-893e08424d43","Type":"ContainerDied","Data":"ac8eac4c10501c2225b337f8910b56b1dc2e2d782270153faf8c218a9ef12094"}
Jul 16 20:37:04.928882 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:04.928854    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb"
Jul 16 20:37:04.999206 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:04.999170    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a260ab4-4a87-491e-81ad-893e08424d43-maas-api-service-ca\") pod \"6a260ab4-4a87-491e-81ad-893e08424d43\" (UID: \"6a260ab4-4a87-491e-81ad-893e08424d43\") "
Jul 16 20:37:04.999584 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:04.999561    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a260ab4-4a87-491e-81ad-893e08424d43-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a260ab4-4a87-491e-81ad-893e08424d43" (UID: "6a260ab4-4a87-491e-81ad-893e08424d43"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:37:05.100096 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:05.099997    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a260ab4-4a87-491e-81ad-893e08424d43-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:37:05.807726 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:05.807691    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb"
Jul 16 20:37:05.807726 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:05.807713    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb" event={"ID":"6a260ab4-4a87-491e-81ad-893e08424d43","Type":"ContainerDied","Data":"5645b41a497906730702bd753d2c91992a838597ae1237090b8835cfb56293e2"}
Jul 16 20:37:05.807994 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:05.807768    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5645b41a497906730702bd753d2c91992a838597ae1237090b8835cfb56293e2"
Jul 16 20:37:12.214311 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.214273    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"]
Jul 16 20:37:12.214795 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.214610    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd" podUID="ce97e411-6495-40e3-a368-69907848cdd0" containerName="maas-api" containerID="cri-o://fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63" gracePeriod=30
Jul 16 20:37:12.471050 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.470978    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:37:12.576459 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.576392    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8rb2\" (UniqueName: \"kubernetes.io/projected/ce97e411-6495-40e3-a368-69907848cdd0-kube-api-access-d8rb2\") pod \"ce97e411-6495-40e3-a368-69907848cdd0\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") "
Jul 16 20:37:12.576647 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.576496    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97e411-6495-40e3-a368-69907848cdd0-maas-api-tls\") pod \"ce97e411-6495-40e3-a368-69907848cdd0\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") "
Jul 16 20:37:12.576647 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.576534    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce97e411-6495-40e3-a368-69907848cdd0-openshift-service-ca\") pod \"ce97e411-6495-40e3-a368-69907848cdd0\" (UID: \"ce97e411-6495-40e3-a368-69907848cdd0\") "
Jul 16 20:37:12.576969 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.576935    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce97e411-6495-40e3-a368-69907848cdd0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce97e411-6495-40e3-a368-69907848cdd0" (UID: "ce97e411-6495-40e3-a368-69907848cdd0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:37:12.578603 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.578579    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce97e411-6495-40e3-a368-69907848cdd0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce97e411-6495-40e3-a368-69907848cdd0" (UID: "ce97e411-6495-40e3-a368-69907848cdd0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:37:12.578688 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.578597    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce97e411-6495-40e3-a368-69907848cdd0-kube-api-access-d8rb2" (OuterVolumeSpecName: "kube-api-access-d8rb2") pod "ce97e411-6495-40e3-a368-69907848cdd0" (UID: "ce97e411-6495-40e3-a368-69907848cdd0"). InnerVolumeSpecName "kube-api-access-d8rb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:37:12.677365 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.677315    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8rb2\" (UniqueName: \"kubernetes.io/projected/ce97e411-6495-40e3-a368-69907848cdd0-kube-api-access-d8rb2\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:37:12.677365 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.677361    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce97e411-6495-40e3-a368-69907848cdd0-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:37:12.677365 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.677372    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce97e411-6495-40e3-a368-69907848cdd0-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:37:12.850361 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.850253    2579 generic.go:356] "Generic (PLEG): container finished" podID="ce97e411-6495-40e3-a368-69907848cdd0" containerID="fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63" exitCode=0
Jul 16 20:37:12.850361 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.850306    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"
Jul 16 20:37:12.850361 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.850339    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd" event={"ID":"ce97e411-6495-40e3-a368-69907848cdd0","Type":"ContainerDied","Data":"fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63"}
Jul 16 20:37:12.850669 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.850411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd" event={"ID":"ce97e411-6495-40e3-a368-69907848cdd0","Type":"ContainerDied","Data":"6441a99151dd82b17075181c3698058b3065b9798b4569a2333f4ff9717b892b"}
Jul 16 20:37:12.850669 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.850461    2579 scope.go:117] "RemoveContainer" containerID="fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63"
Jul 16 20:37:12.860537 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.860515    2579 scope.go:117] "RemoveContainer" containerID="fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63"
Jul 16 20:37:12.860807 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:37:12.860788    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63\": container with ID starting with fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63 not found: ID does not exist" containerID="fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63"
Jul 16 20:37:12.860847 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.860816    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63"} err="failed to get container status \"fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63\": rpc error: code = NotFound desc = could not find container \"fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63\": container with ID starting with fb0776d1d279868975c3388e36d5dd496ae8d6303179983c1de5f9aeda7e8c63 not found: ID does not exist"
Jul 16 20:37:12.872981 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.872941    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"]
Jul 16 20:37:12.876837 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:12.876797    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0faadf3f-86dd77b774-d58nd"]
Jul 16 20:37:14.405290 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:14.405256    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce97e411-6495-40e3-a368-69907848cdd0" path="/var/lib/kubelet/pods/ce97e411-6495-40e3-a368-69907848cdd0/volumes"
Jul 16 20:37:15.784466 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:15.784416    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:37:36.554894 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:37:36.554850    2579 scope.go:117] "RemoveContainer" containerID="9b43f6c3d5c7600df332d2361102fb4bfc9f034fb689b615cbc223a6fbcd87b8"
Jul 16 20:38:19.289802 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:38:19.289758    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:38:28.987614 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:38:28.987565    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:39:06.396949 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.396909    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"]
Jul 16 20:39:06.397592 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.397551    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a260ab4-4a87-491e-81ad-893e08424d43" containerName="revoke-keys"
Jul 16 20:39:06.397592 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.397574    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a260ab4-4a87-491e-81ad-893e08424d43" containerName="revoke-keys"
Jul 16 20:39:06.397592 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.397593    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce97e411-6495-40e3-a368-69907848cdd0" containerName="maas-api"
Jul 16 20:39:06.397776 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.397602    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce97e411-6495-40e3-a368-69907848cdd0" containerName="maas-api"
Jul 16 20:39:06.397776 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.397711    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a260ab4-4a87-491e-81ad-893e08424d43" containerName="revoke-keys"
Jul 16 20:39:06.397776 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.397725    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce97e411-6495-40e3-a368-69907848cdd0" containerName="maas-api"
Jul 16 20:39:06.401047 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.401023    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:06.403389 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.403358    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-955113-serving-cert\""
Jul 16 20:39:06.403558 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.403368    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 20:39:06.403558 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.403538    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g78x2\""
Jul 16 20:39:06.404156 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.404133    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 20:39:06.412266 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.412241    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"]
Jul 16 20:39:06.522747 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.522706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ca653e-dba1-419b-b608-67ec2200208c-maas-api-tls\") pod \"maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:06.522960 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.522765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ca653e-dba1-419b-b608-67ec2200208c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:06.522960 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.522835    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k59dm\" (UniqueName: \"kubernetes.io/projected/61ca653e-dba1-419b-b608-67ec2200208c-kube-api-access-k59dm\") pod \"maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:06.623821 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.623776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k59dm\" (UniqueName: \"kubernetes.io/projected/61ca653e-dba1-419b-b608-67ec2200208c-kube-api-access-k59dm\") pod \"maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:06.624035 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.623880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ca653e-dba1-419b-b608-67ec2200208c-maas-api-tls\") pod \"maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:06.624035 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.623926    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ca653e-dba1-419b-b608-67ec2200208c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:06.624174 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:39:06.624035    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-serving-cert: secret "maas-api-e2e-inf-a-955113-serving-cert" not found
Jul 16 20:39:06.624174 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:39:06.624119    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61ca653e-dba1-419b-b608-67ec2200208c-maas-api-tls podName:61ca653e-dba1-419b-b608-67ec2200208c nodeName:}" failed. No retries permitted until 2026-07-16 20:39:07.124095823 +0000 UTC m=+2311.272191241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/61ca653e-dba1-419b-b608-67ec2200208c-maas-api-tls") pod "maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct" (UID: "61ca653e-dba1-419b-b608-67ec2200208c") : secret "maas-api-e2e-inf-a-955113-serving-cert" not found
Jul 16 20:39:06.624656 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.624633    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ca653e-dba1-419b-b608-67ec2200208c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:06.632691 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:06.632667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k59dm\" (UniqueName: \"kubernetes.io/projected/61ca653e-dba1-419b-b608-67ec2200208c-kube-api-access-k59dm\") pod \"maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:07.129023 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:07.128985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ca653e-dba1-419b-b608-67ec2200208c-maas-api-tls\") pod \"maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:07.131461 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:07.131425    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ca653e-dba1-419b-b608-67ec2200208c-maas-api-tls\") pod \"maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:07.314772 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:07.314723    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:07.459125 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:07.459096    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"]
Jul 16 20:39:07.461334 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:39:07.461295    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ca653e_dba1_419b_b608_67ec2200208c.slice/crio-14241edabd532bb4bea4a6f730682b6ae0a35899bcd2b392643da1f922796ef1 WatchSource:0}: Error finding container 14241edabd532bb4bea4a6f730682b6ae0a35899bcd2b392643da1f922796ef1: Status 404 returned error can't find the container with id 14241edabd532bb4bea4a6f730682b6ae0a35899bcd2b392643da1f922796ef1
Jul 16 20:39:07.516253 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:07.516212    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct" event={"ID":"61ca653e-dba1-419b-b608-67ec2200208c","Type":"ContainerStarted","Data":"14241edabd532bb4bea4a6f730682b6ae0a35899bcd2b392643da1f922796ef1"}
Jul 16 20:39:08.523916 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:08.523879    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct" event={"ID":"61ca653e-dba1-419b-b608-67ec2200208c","Type":"ContainerStarted","Data":"972fda217648fb1b9e683b5df51a7db7e0d51db01ca298c13b0698cb48767448"}
Jul 16 20:39:08.524459 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:08.523942    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:08.544200 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:08.544135    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct" podStartSLOduration=2.109158514 podStartE2EDuration="2.544118305s" totalImagesPullingTime="434.959791ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:39:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:39:08.540633298 +0000 UTC m=+2312.688728748" watchObservedRunningTime="2026-07-16 20:39:08.544118305 +0000 UTC m=+2312.692213727"
Jul 16 20:39:12.290448 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:12.290392    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:39:14.289425 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.289383    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"]
Jul 16 20:39:14.293416 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.293386    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:14.298151 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.298127    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e69c2e-serving-cert\""
Jul 16 20:39:14.306308 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.306276    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"]
Jul 16 20:39:14.405734 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.405703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f829ab02-2158-48b1-b256-d2cd035e0cac-maas-api-tls\") pod \"maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:14.405936 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.405766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk5k8\" (UniqueName: \"kubernetes.io/projected/f829ab02-2158-48b1-b256-d2cd035e0cac-kube-api-access-pk5k8\") pod \"maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:14.405936 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.405820    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f829ab02-2158-48b1-b256-d2cd035e0cac-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:14.507312 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.507264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f829ab02-2158-48b1-b256-d2cd035e0cac-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:14.507583 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.507349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f829ab02-2158-48b1-b256-d2cd035e0cac-maas-api-tls\") pod \"maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:14.507583 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.507409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk5k8\" (UniqueName: \"kubernetes.io/projected/f829ab02-2158-48b1-b256-d2cd035e0cac-kube-api-access-pk5k8\") pod \"maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:14.507583 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:39:14.507526    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-serving-cert: secret "maas-api-e2e-inf-b-e69c2e-serving-cert" not found
Jul 16 20:39:14.507783 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:39:14.507617    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f829ab02-2158-48b1-b256-d2cd035e0cac-maas-api-tls podName:f829ab02-2158-48b1-b256-d2cd035e0cac nodeName:}" failed. No retries permitted until 2026-07-16 20:39:15.007590237 +0000 UTC m=+2319.155685657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f829ab02-2158-48b1-b256-d2cd035e0cac-maas-api-tls") pod "maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh" (UID: "f829ab02-2158-48b1-b256-d2cd035e0cac") : secret "maas-api-e2e-inf-b-e69c2e-serving-cert" not found
Jul 16 20:39:14.508076 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.508049    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f829ab02-2158-48b1-b256-d2cd035e0cac-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:14.529552 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.529506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk5k8\" (UniqueName: \"kubernetes.io/projected/f829ab02-2158-48b1-b256-d2cd035e0cac-kube-api-access-pk5k8\") pod \"maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:14.535402 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:14.535374    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:39:15.013590 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:15.013215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f829ab02-2158-48b1-b256-d2cd035e0cac-maas-api-tls\") pod \"maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:15.016856 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:15.016816    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f829ab02-2158-48b1-b256-d2cd035e0cac-maas-api-tls\") pod \"maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:15.206233 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:15.206191    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:15.551498 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:15.551466    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"]
Jul 16 20:39:15.552912 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:39:15.552882    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf829ab02_2158_48b1_b256_d2cd035e0cac.slice/crio-c4c457860aedc644db79ff1c1bb8cc83cde7c516907a23170e621a7b2a7fc0cf WatchSource:0}: Error finding container c4c457860aedc644db79ff1c1bb8cc83cde7c516907a23170e621a7b2a7fc0cf: Status 404 returned error can't find the container with id c4c457860aedc644db79ff1c1bb8cc83cde7c516907a23170e621a7b2a7fc0cf
Jul 16 20:39:15.566148 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:15.566111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh" event={"ID":"f829ab02-2158-48b1-b256-d2cd035e0cac","Type":"ContainerStarted","Data":"c4c457860aedc644db79ff1c1bb8cc83cde7c516907a23170e621a7b2a7fc0cf"}
Jul 16 20:39:16.573593 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:16.573540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh" event={"ID":"f829ab02-2158-48b1-b256-d2cd035e0cac","Type":"ContainerStarted","Data":"332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1"}
Jul 16 20:39:16.574052 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:16.573691    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:16.590247 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:16.590186    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh" podStartSLOduration=2.250262891 podStartE2EDuration="2.590167383s" totalImagesPullingTime="339.904492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:39:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:39:16.588565287 +0000 UTC m=+2320.736660732" watchObservedRunningTime="2026-07-16 20:39:16.590167383 +0000 UTC m=+2320.738262806"
Jul 16 20:39:22.584805 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:22.584772    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:39:26.083225 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:26.083178    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:39:36.590125 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:36.590089    2579 scope.go:117] "RemoveContainer" containerID="96f876608435a841846badff9bb5faf31e3c421afdcef69bcae80786144225c5"
Jul 16 20:39:44.682705 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:44.682666    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:39:55.979129 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:39:55.979092    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:40:20.988039 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:20.987996    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:40:34.379665 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:34.379619    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59"]
Jul 16 20:40:34.383565 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:34.383536    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59"
Jul 16 20:40:34.385942 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:34.385918    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:40:34.391571 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:34.391543    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59"]
Jul 16 20:40:34.447796 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:34.447756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdb14d-54bd-4219-838e-eba2f815bc21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-955113-spn59\" (UID: \"e8bdb14d-54bd-4219-838e-eba2f815bc21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59"
Jul 16 20:40:34.549023 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:34.548975    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdb14d-54bd-4219-838e-eba2f815bc21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-955113-spn59\" (UID: \"e8bdb14d-54bd-4219-838e-eba2f815bc21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59"
Jul 16 20:40:34.549790 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:34.549767    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdb14d-54bd-4219-838e-eba2f815bc21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-955113-spn59\" (UID: \"e8bdb14d-54bd-4219-838e-eba2f815bc21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59"
Jul 16 20:40:34.696635 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:34.696595    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59"
Jul 16 20:40:34.833244 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:34.833216    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59"]
Jul 16 20:40:34.835540 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:40:34.835507    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8bdb14d_54bd_4219_838e_eba2f815bc21.slice/crio-03a5e5ca7b9d403de96a2c22e22a818a8688c741ceb18f9e072f2d5886af88a9 WatchSource:0}: Error finding container 03a5e5ca7b9d403de96a2c22e22a818a8688c741ceb18f9e072f2d5886af88a9: Status 404 returned error can't find the container with id 03a5e5ca7b9d403de96a2c22e22a818a8688c741ceb18f9e072f2d5886af88a9
Jul 16 20:40:35.025008 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:35.024980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59" event={"ID":"e8bdb14d-54bd-4219-838e-eba2f815bc21","Type":"ContainerStarted","Data":"a59eb63e800dd4b9b5930bb22308198a683b5c03e017c1ac8d3064e3fd95398c"}
Jul 16 20:40:35.025144 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:35.025030    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59" event={"ID":"e8bdb14d-54bd-4219-838e-eba2f815bc21","Type":"ContainerStarted","Data":"03a5e5ca7b9d403de96a2c22e22a818a8688c741ceb18f9e072f2d5886af88a9"}
Jul 16 20:40:35.042420 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:35.042368    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59" podStartSLOduration=1.042353582 podStartE2EDuration="1.042353582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:40:35.039248299 +0000 UTC m=+2399.187343718" watchObservedRunningTime="2026-07-16 20:40:35.042353582 +0000 UTC m=+2399.190449044"
Jul 16 20:40:36.032094 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:36.032060    2579 generic.go:356] "Generic (PLEG): container finished" podID="e8bdb14d-54bd-4219-838e-eba2f815bc21" containerID="a59eb63e800dd4b9b5930bb22308198a683b5c03e017c1ac8d3064e3fd95398c" exitCode=0
Jul 16 20:40:36.032618 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:36.032160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59" event={"ID":"e8bdb14d-54bd-4219-838e-eba2f815bc21","Type":"ContainerDied","Data":"a59eb63e800dd4b9b5930bb22308198a683b5c03e017c1ac8d3064e3fd95398c"}
Jul 16 20:40:37.172872 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:37.172842    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59"
Jul 16 20:40:37.275716 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:37.275674    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdb14d-54bd-4219-838e-eba2f815bc21-maas-api-service-ca\") pod \"e8bdb14d-54bd-4219-838e-eba2f815bc21\" (UID: \"e8bdb14d-54bd-4219-838e-eba2f815bc21\") "
Jul 16 20:40:37.276050 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:37.276026    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8bdb14d-54bd-4219-838e-eba2f815bc21-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8bdb14d-54bd-4219-838e-eba2f815bc21" (UID: "e8bdb14d-54bd-4219-838e-eba2f815bc21"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:40:37.376851 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:37.376764    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdb14d-54bd-4219-838e-eba2f815bc21-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:40:38.045423 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:38.045383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59" event={"ID":"e8bdb14d-54bd-4219-838e-eba2f815bc21","Type":"ContainerDied","Data":"03a5e5ca7b9d403de96a2c22e22a818a8688c741ceb18f9e072f2d5886af88a9"}
Jul 16 20:40:38.045423 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:38.045415    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-955113-spn59"
Jul 16 20:40:38.045670 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:38.045459    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03a5e5ca7b9d403de96a2c22e22a818a8688c741ceb18f9e072f2d5886af88a9"
Jul 16 20:40:39.389363 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:39.389326    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:40:44.902882 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:44.902835    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"]
Jul 16 20:40:44.903487 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:44.903099    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct" podUID="61ca653e-dba1-419b-b608-67ec2200208c" containerName="maas-api" containerID="cri-o://972fda217648fb1b9e683b5df51a7db7e0d51db01ca298c13b0698cb48767448" gracePeriod=30
Jul 16 20:40:45.097654 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.097614    2579 generic.go:356] "Generic (PLEG): container finished" podID="61ca653e-dba1-419b-b608-67ec2200208c" containerID="972fda217648fb1b9e683b5df51a7db7e0d51db01ca298c13b0698cb48767448" exitCode=0
Jul 16 20:40:45.097829 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.097714    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct" event={"ID":"61ca653e-dba1-419b-b608-67ec2200208c","Type":"ContainerDied","Data":"972fda217648fb1b9e683b5df51a7db7e0d51db01ca298c13b0698cb48767448"}
Jul 16 20:40:45.163289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.163270    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:40:45.350750 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.350695    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ca653e-dba1-419b-b608-67ec2200208c-maas-api-tls\") pod \"61ca653e-dba1-419b-b608-67ec2200208c\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") "
Jul 16 20:40:45.350947 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.350786    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ca653e-dba1-419b-b608-67ec2200208c-openshift-service-ca\") pod \"61ca653e-dba1-419b-b608-67ec2200208c\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") "
Jul 16 20:40:45.350947 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.350818    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k59dm\" (UniqueName: \"kubernetes.io/projected/61ca653e-dba1-419b-b608-67ec2200208c-kube-api-access-k59dm\") pod \"61ca653e-dba1-419b-b608-67ec2200208c\" (UID: \"61ca653e-dba1-419b-b608-67ec2200208c\") "
Jul 16 20:40:45.351241 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.351198    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ca653e-dba1-419b-b608-67ec2200208c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "61ca653e-dba1-419b-b608-67ec2200208c" (UID: "61ca653e-dba1-419b-b608-67ec2200208c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:40:45.352907 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.352886    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61ca653e-dba1-419b-b608-67ec2200208c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "61ca653e-dba1-419b-b608-67ec2200208c" (UID: "61ca653e-dba1-419b-b608-67ec2200208c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:40:45.353108 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.353093    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61ca653e-dba1-419b-b608-67ec2200208c-kube-api-access-k59dm" (OuterVolumeSpecName: "kube-api-access-k59dm") pod "61ca653e-dba1-419b-b608-67ec2200208c" (UID: "61ca653e-dba1-419b-b608-67ec2200208c"). InnerVolumeSpecName "kube-api-access-k59dm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:40:45.452550 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.452423    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61ca653e-dba1-419b-b608-67ec2200208c-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:40:45.452550 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.452496    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61ca653e-dba1-419b-b608-67ec2200208c-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:40:45.452550 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:45.452507    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k59dm\" (UniqueName: \"kubernetes.io/projected/61ca653e-dba1-419b-b608-67ec2200208c-kube-api-access-k59dm\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:40:46.104678 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:46.104638    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct" event={"ID":"61ca653e-dba1-419b-b608-67ec2200208c","Type":"ContainerDied","Data":"14241edabd532bb4bea4a6f730682b6ae0a35899bcd2b392643da1f922796ef1"}
Jul 16 20:40:46.104678 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:46.104671    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"
Jul 16 20:40:46.105176 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:46.104715    2579 scope.go:117] "RemoveContainer" containerID="972fda217648fb1b9e683b5df51a7db7e0d51db01ca298c13b0698cb48767448"
Jul 16 20:40:46.127103 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:46.127072    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"]
Jul 16 20:40:46.131950 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:46.131926    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-955113-6c4bc88cc5-kqgct"]
Jul 16 20:40:46.405610 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:46.405531    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61ca653e-dba1-419b-b608-67ec2200208c" path="/var/lib/kubelet/pods/61ca653e-dba1-419b-b608-67ec2200208c/volumes"
Jul 16 20:40:49.299973 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:40:49.299932    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:41:00.584905 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.584819    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424"]
Jul 16 20:41:00.585412 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.585208    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61ca653e-dba1-419b-b608-67ec2200208c" containerName="maas-api"
Jul 16 20:41:00.585412 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.585220    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="61ca653e-dba1-419b-b608-67ec2200208c" containerName="maas-api"
Jul 16 20:41:00.585412 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.585228    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8bdb14d-54bd-4219-838e-eba2f815bc21" containerName="revoke-keys"
Jul 16 20:41:00.585412 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.585233    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8bdb14d-54bd-4219-838e-eba2f815bc21" containerName="revoke-keys"
Jul 16 20:41:00.585412 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.585289    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="61ca653e-dba1-419b-b608-67ec2200208c" containerName="maas-api"
Jul 16 20:41:00.585412 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.585300    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8bdb14d-54bd-4219-838e-eba2f815bc21" containerName="revoke-keys"
Jul 16 20:41:00.589679 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.589662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424"
Jul 16 20:41:00.592160 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.592131    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:41:00.594319 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.594291    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424"]
Jul 16 20:41:00.678973 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.678939    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb73860-9f1a-4c9d-8546-be9ec64feb55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424\" (UID: \"fdb73860-9f1a-4c9d-8546-be9ec64feb55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424"
Jul 16 20:41:00.780047 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.780005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb73860-9f1a-4c9d-8546-be9ec64feb55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424\" (UID: \"fdb73860-9f1a-4c9d-8546-be9ec64feb55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424"
Jul 16 20:41:00.780675 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.780652    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb73860-9f1a-4c9d-8546-be9ec64feb55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424\" (UID: \"fdb73860-9f1a-4c9d-8546-be9ec64feb55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424"
Jul 16 20:41:00.901698 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:00.901591    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424"
Jul 16 20:41:01.057853 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:01.057822    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424"]
Jul 16 20:41:01.059092 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:41:01.059062    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb73860_9f1a_4c9d_8546_be9ec64feb55.slice/crio-48dd259fc22ce87b3ff2cd04e2c594413f6ae2ee35e8ac0bb8f341ed27d0c808 WatchSource:0}: Error finding container 48dd259fc22ce87b3ff2cd04e2c594413f6ae2ee35e8ac0bb8f341ed27d0c808: Status 404 returned error can't find the container with id 48dd259fc22ce87b3ff2cd04e2c594413f6ae2ee35e8ac0bb8f341ed27d0c808
Jul 16 20:41:01.199125 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:01.199074    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424" event={"ID":"fdb73860-9f1a-4c9d-8546-be9ec64feb55","Type":"ContainerStarted","Data":"48dd259fc22ce87b3ff2cd04e2c594413f6ae2ee35e8ac0bb8f341ed27d0c808"}
Jul 16 20:41:02.205865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:02.205832    2579 generic.go:356] "Generic (PLEG): container finished" podID="fdb73860-9f1a-4c9d-8546-be9ec64feb55" containerID="1275c7120d8fd2910d53b3f1a81ba457ec745b0f6b0e6541f1287ae46b75cd57" exitCode=0
Jul 16 20:41:02.206315 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:02.205927    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424" event={"ID":"fdb73860-9f1a-4c9d-8546-be9ec64feb55","Type":"ContainerDied","Data":"1275c7120d8fd2910d53b3f1a81ba457ec745b0f6b0e6541f1287ae46b75cd57"}
Jul 16 20:41:03.348654 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:03.348623    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424"
Jul 16 20:41:03.402677 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:03.402638    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb73860-9f1a-4c9d-8546-be9ec64feb55-maas-api-service-ca\") pod \"fdb73860-9f1a-4c9d-8546-be9ec64feb55\" (UID: \"fdb73860-9f1a-4c9d-8546-be9ec64feb55\") "
Jul 16 20:41:03.403008 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:03.402983    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdb73860-9f1a-4c9d-8546-be9ec64feb55-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fdb73860-9f1a-4c9d-8546-be9ec64feb55" (UID: "fdb73860-9f1a-4c9d-8546-be9ec64feb55"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:41:03.504158 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:03.504055    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb73860-9f1a-4c9d-8546-be9ec64feb55-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:41:04.218099 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:04.218066    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424"
Jul 16 20:41:04.218281 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:04.218108    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424" event={"ID":"fdb73860-9f1a-4c9d-8546-be9ec64feb55","Type":"ContainerDied","Data":"48dd259fc22ce87b3ff2cd04e2c594413f6ae2ee35e8ac0bb8f341ed27d0c808"}
Jul 16 20:41:04.218281 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:04.218157    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48dd259fc22ce87b3ff2cd04e2c594413f6ae2ee35e8ac0bb8f341ed27d0c808"
Jul 16 20:41:10.992829 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:10.992795    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"]
Jul 16 20:41:10.993575 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:10.993046    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh" podUID="f829ab02-2158-48b1-b256-d2cd035e0cac" containerName="maas-api" containerID="cri-o://332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1" gracePeriod=30
Jul 16 20:41:11.463568 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:11.463536    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:41:11.582573 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:11.582473    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk5k8\" (UniqueName: \"kubernetes.io/projected/f829ab02-2158-48b1-b256-d2cd035e0cac-kube-api-access-pk5k8\") pod \"f829ab02-2158-48b1-b256-d2cd035e0cac\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") "
Jul 16 20:41:11.582751 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:11.582619    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f829ab02-2158-48b1-b256-d2cd035e0cac-maas-api-tls\") pod \"f829ab02-2158-48b1-b256-d2cd035e0cac\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") "
Jul 16 20:41:11.582751 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:11.582653    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f829ab02-2158-48b1-b256-d2cd035e0cac-openshift-service-ca\") pod \"f829ab02-2158-48b1-b256-d2cd035e0cac\" (UID: \"f829ab02-2158-48b1-b256-d2cd035e0cac\") "
Jul 16 20:41:11.583028 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:11.582997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f829ab02-2158-48b1-b256-d2cd035e0cac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f829ab02-2158-48b1-b256-d2cd035e0cac" (UID: "f829ab02-2158-48b1-b256-d2cd035e0cac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:41:11.584630 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:11.584607    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f829ab02-2158-48b1-b256-d2cd035e0cac-kube-api-access-pk5k8" (OuterVolumeSpecName: "kube-api-access-pk5k8") pod "f829ab02-2158-48b1-b256-d2cd035e0cac" (UID: "f829ab02-2158-48b1-b256-d2cd035e0cac"). InnerVolumeSpecName "kube-api-access-pk5k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:41:11.584724 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:11.584652    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f829ab02-2158-48b1-b256-d2cd035e0cac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f829ab02-2158-48b1-b256-d2cd035e0cac" (UID: "f829ab02-2158-48b1-b256-d2cd035e0cac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:41:11.684117 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:11.684073    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f829ab02-2158-48b1-b256-d2cd035e0cac-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:41:11.684117 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:11.684110    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f829ab02-2158-48b1-b256-d2cd035e0cac-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:41:11.684117 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:11.684131    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk5k8\" (UniqueName: \"kubernetes.io/projected/f829ab02-2158-48b1-b256-d2cd035e0cac-kube-api-access-pk5k8\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:41:12.266523 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:12.266490    2579 generic.go:356] "Generic (PLEG): container finished" podID="f829ab02-2158-48b1-b256-d2cd035e0cac" containerID="332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1" exitCode=0
Jul 16 20:41:12.266963 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:12.266544    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"
Jul 16 20:41:12.266963 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:12.266586    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh" event={"ID":"f829ab02-2158-48b1-b256-d2cd035e0cac","Type":"ContainerDied","Data":"332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1"}
Jul 16 20:41:12.266963 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:12.266648    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh" event={"ID":"f829ab02-2158-48b1-b256-d2cd035e0cac","Type":"ContainerDied","Data":"c4c457860aedc644db79ff1c1bb8cc83cde7c516907a23170e621a7b2a7fc0cf"}
Jul 16 20:41:12.266963 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:12.266675    2579 scope.go:117] "RemoveContainer" containerID="332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1"
Jul 16 20:41:12.276865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:12.276843    2579 scope.go:117] "RemoveContainer" containerID="332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1"
Jul 16 20:41:12.277171 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:41:12.277147    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1\": container with ID starting with 332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1 not found: ID does not exist" containerID="332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1"
Jul 16 20:41:12.277289 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:12.277178    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1"} err="failed to get container status \"332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1\": rpc error: code = NotFound desc = could not find container \"332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1\": container with ID starting with 332a035586621dba8aab5441ba8c76f45b183b6bfe6f9e3a187e0d23a747c3d1 not found: ID does not exist"
Jul 16 20:41:12.288499 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:12.288471    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"]
Jul 16 20:41:12.292643 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:12.292617    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e69c2e-b5b44655d-bbdrh"]
Jul 16 20:41:12.404692 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:12.404660    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f829ab02-2158-48b1-b256-d2cd035e0cac" path="/var/lib/kubelet/pods/f829ab02-2158-48b1-b256-d2cd035e0cac/volumes"
Jul 16 20:41:14.180100 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:14.180061    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:41:29.896985 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.896943    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"]
Jul 16 20:41:29.899399 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.897331    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f829ab02-2158-48b1-b256-d2cd035e0cac" containerName="maas-api"
Jul 16 20:41:29.899399 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.897342    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f829ab02-2158-48b1-b256-d2cd035e0cac" containerName="maas-api"
Jul 16 20:41:29.899399 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.897355    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdb73860-9f1a-4c9d-8546-be9ec64feb55" containerName="revoke-keys"
Jul 16 20:41:29.899399 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.897361    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdb73860-9f1a-4c9d-8546-be9ec64feb55" containerName="revoke-keys"
Jul 16 20:41:29.899399 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.897418    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdb73860-9f1a-4c9d-8546-be9ec64feb55" containerName="revoke-keys"
Jul 16 20:41:29.899399 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.897427    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f829ab02-2158-48b1-b256-d2cd035e0cac" containerName="maas-api"
Jul 16 20:41:29.900463 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.900419    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:29.903041 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.903009    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 20:41:29.903190 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.903045    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g78x2\""
Jul 16 20:41:29.903667 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.903646    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-7cfc57-serving-cert\""
Jul 16 20:41:29.903733 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.903653    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 20:41:29.910590 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:29.910562    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"]
Jul 16 20:41:30.051811 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.051762    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fss8v\" (UniqueName: \"kubernetes.io/projected/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-kube-api-access-fss8v\") pod \"maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.052017 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.051827    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-maas-api-tls\") pod \"maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.052017 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.051856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-openshift-service-ca\") pod \"maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.152853 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.152761    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-maas-api-tls\") pod \"maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.152853 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.152810    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-openshift-service-ca\") pod \"maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.153064 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:41:30.152915    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-serving-cert: secret "maas-api-e2e-api-a-7cfc57-serving-cert" not found
Jul 16 20:41:30.153064 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.152929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fss8v\" (UniqueName: \"kubernetes.io/projected/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-kube-api-access-fss8v\") pod \"maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.153064 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:41:30.152989    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-maas-api-tls podName:f5ce85f5-6f1b-4466-815f-de6ceb5e9612 nodeName:}" failed. No retries permitted until 2026-07-16 20:41:30.652969933 +0000 UTC m=+2454.801065337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-maas-api-tls") pod "maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d" (UID: "f5ce85f5-6f1b-4466-815f-de6ceb5e9612") : secret "maas-api-e2e-api-a-7cfc57-serving-cert" not found
Jul 16 20:41:30.153567 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.153546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-openshift-service-ca\") pod \"maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.174792 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.174751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fss8v\" (UniqueName: \"kubernetes.io/projected/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-kube-api-access-fss8v\") pod \"maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.658365 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.658321    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-maas-api-tls\") pod \"maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.660885 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.660854    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-maas-api-tls\") pod \"maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.813202 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.813156    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:30.958628 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.958594    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"]
Jul 16 20:41:30.960483 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:41:30.960458    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ce85f5_6f1b_4466_815f_de6ceb5e9612.slice/crio-7ecc1861916aaf509ca5c4248f2284ffa3d01f9b3914e2c4585ff1523a389abe WatchSource:0}: Error finding container 7ecc1861916aaf509ca5c4248f2284ffa3d01f9b3914e2c4585ff1523a389abe: Status 404 returned error can't find the container with id 7ecc1861916aaf509ca5c4248f2284ffa3d01f9b3914e2c4585ff1523a389abe
Jul 16 20:41:30.962196 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:30.962178    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 20:41:31.382414 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:31.382329    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d" event={"ID":"f5ce85f5-6f1b-4466-815f-de6ceb5e9612","Type":"ContainerStarted","Data":"7ecc1861916aaf509ca5c4248f2284ffa3d01f9b3914e2c4585ff1523a389abe"}
Jul 16 20:41:32.389920 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:32.389881    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d" event={"ID":"f5ce85f5-6f1b-4466-815f-de6ceb5e9612","Type":"ContainerStarted","Data":"f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033"}
Jul 16 20:41:32.390352 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:32.389947    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:32.410375 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:32.410291    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d" podStartSLOduration=2.695365015 podStartE2EDuration="3.410270767s" totalImagesPullingTime="714.905752ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:41:32.408594516 +0000 UTC m=+2456.556689960" watchObservedRunningTime="2026-07-16 20:41:32.410270767 +0000 UTC m=+2456.558366192"
Jul 16 20:41:34.899611 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:34.899572    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:41:38.004207 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.004160    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"]
Jul 16 20:41:38.009863 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.009839    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.012234 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.012210    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-1dad58-serving-cert\""
Jul 16 20:41:38.019613 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.019580    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"]
Jul 16 20:41:38.127382 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.127332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv8jg\" (UniqueName: \"kubernetes.io/projected/97953106-7061-474f-99b3-304c6376a761-kube-api-access-pv8jg\") pod \"maas-api-e2e-api-b-1dad58-f787d7976-9t6p5\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.127621 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.127404    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97953106-7061-474f-99b3-304c6376a761-maas-api-tls\") pod \"maas-api-e2e-api-b-1dad58-f787d7976-9t6p5\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.127621 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.127481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97953106-7061-474f-99b3-304c6376a761-openshift-service-ca\") pod \"maas-api-e2e-api-b-1dad58-f787d7976-9t6p5\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.228471 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.228421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv8jg\" (UniqueName: \"kubernetes.io/projected/97953106-7061-474f-99b3-304c6376a761-kube-api-access-pv8jg\") pod \"maas-api-e2e-api-b-1dad58-f787d7976-9t6p5\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.228671 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.228513    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97953106-7061-474f-99b3-304c6376a761-maas-api-tls\") pod \"maas-api-e2e-api-b-1dad58-f787d7976-9t6p5\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.228671 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.228562    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97953106-7061-474f-99b3-304c6376a761-openshift-service-ca\") pod \"maas-api-e2e-api-b-1dad58-f787d7976-9t6p5\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.228671 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:41:38.228657    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-serving-cert: secret "maas-api-e2e-api-b-1dad58-serving-cert" not found
Jul 16 20:41:38.228850 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:41:38.228737    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97953106-7061-474f-99b3-304c6376a761-maas-api-tls podName:97953106-7061-474f-99b3-304c6376a761 nodeName:}" failed. No retries permitted until 2026-07-16 20:41:38.728721585 +0000 UTC m=+2462.876816986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/97953106-7061-474f-99b3-304c6376a761-maas-api-tls") pod "maas-api-e2e-api-b-1dad58-f787d7976-9t6p5" (UID: "97953106-7061-474f-99b3-304c6376a761") : secret "maas-api-e2e-api-b-1dad58-serving-cert" not found
Jul 16 20:41:38.229273 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.229249    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97953106-7061-474f-99b3-304c6376a761-openshift-service-ca\") pod \"maas-api-e2e-api-b-1dad58-f787d7976-9t6p5\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.237578 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.237553    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv8jg\" (UniqueName: \"kubernetes.io/projected/97953106-7061-474f-99b3-304c6376a761-kube-api-access-pv8jg\") pod \"maas-api-e2e-api-b-1dad58-f787d7976-9t6p5\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.405855 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.405780    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:41:38.733843 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.733799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97953106-7061-474f-99b3-304c6376a761-maas-api-tls\") pod \"maas-api-e2e-api-b-1dad58-f787d7976-9t6p5\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.736377 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.736354    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97953106-7061-474f-99b3-304c6376a761-maas-api-tls\") pod \"maas-api-e2e-api-b-1dad58-f787d7976-9t6p5\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:38.922609 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:38.922554    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:39.061829 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:39.061799    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"]
Jul 16 20:41:39.063622 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:41:39.063599    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97953106_7061_474f_99b3_304c6376a761.slice/crio-3f6021b7d7f2ce4b55a26d398d827e9afd0b2c5526110502924c10d794ec9ec9 WatchSource:0}: Error finding container 3f6021b7d7f2ce4b55a26d398d827e9afd0b2c5526110502924c10d794ec9ec9: Status 404 returned error can't find the container with id 3f6021b7d7f2ce4b55a26d398d827e9afd0b2c5526110502924c10d794ec9ec9
Jul 16 20:41:39.433403 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:39.433372    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5" event={"ID":"97953106-7061-474f-99b3-304c6376a761","Type":"ContainerStarted","Data":"612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0"}
Jul 16 20:41:39.433565 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:39.433424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5" event={"ID":"97953106-7061-474f-99b3-304c6376a761","Type":"ContainerStarted","Data":"3f6021b7d7f2ce4b55a26d398d827e9afd0b2c5526110502924c10d794ec9ec9"}
Jul 16 20:41:39.433565 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:39.433490    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:39.465896 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:39.465842    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5" podStartSLOduration=2.194197691 podStartE2EDuration="2.465824797s" totalImagesPullingTime="271.627106ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:41:39.463819742 +0000 UTC m=+2463.611915178" watchObservedRunningTime="2026-07-16 20:41:39.465824797 +0000 UTC m=+2463.613920213"
Jul 16 20:41:45.444278 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:45.444248    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:41:45.810953 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:45.810862    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:41:50.392427 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:50.392392    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd"]
Jul 16 20:41:50.396301 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:50.396283    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd"
Jul 16 20:41:50.398466 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:50.398442    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:41:50.406850 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:50.406821    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd"]
Jul 16 20:41:50.434043 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:50.434008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/babed4f2-aecb-4814-8da3-57fecb4617f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd\" (UID: \"babed4f2-aecb-4814-8da3-57fecb4617f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd"
Jul 16 20:41:50.535273 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:50.535238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/babed4f2-aecb-4814-8da3-57fecb4617f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd\" (UID: \"babed4f2-aecb-4814-8da3-57fecb4617f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd"
Jul 16 20:41:50.535923 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:50.535901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/babed4f2-aecb-4814-8da3-57fecb4617f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd\" (UID: \"babed4f2-aecb-4814-8da3-57fecb4617f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd"
Jul 16 20:41:50.709590 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:50.709554    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd"
Jul 16 20:41:50.849487 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:50.849454    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd"]
Jul 16 20:41:50.852364 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:41:50.852336    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbabed4f2_aecb_4814_8da3_57fecb4617f6.slice/crio-c406ffe06bbc58b5ed2cffc5b2f5d791baf0a6ca965e16ce49e8b650fbfdafdd WatchSource:0}: Error finding container c406ffe06bbc58b5ed2cffc5b2f5d791baf0a6ca965e16ce49e8b650fbfdafdd: Status 404 returned error can't find the container with id c406ffe06bbc58b5ed2cffc5b2f5d791baf0a6ca965e16ce49e8b650fbfdafdd
Jul 16 20:41:51.506304 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:51.506270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd" event={"ID":"babed4f2-aecb-4814-8da3-57fecb4617f6","Type":"ContainerStarted","Data":"b9e0e342a7d954e609631a5c89e7a7bac801bfd50f18c9ffee7315a516cee57b"}
Jul 16 20:41:51.506631 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:51.506331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd" event={"ID":"babed4f2-aecb-4814-8da3-57fecb4617f6","Type":"ContainerStarted","Data":"c406ffe06bbc58b5ed2cffc5b2f5d791baf0a6ca965e16ce49e8b650fbfdafdd"}
Jul 16 20:41:52.514191 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:52.514155    2579 generic.go:356] "Generic (PLEG): container finished" podID="babed4f2-aecb-4814-8da3-57fecb4617f6" containerID="b9e0e342a7d954e609631a5c89e7a7bac801bfd50f18c9ffee7315a516cee57b" exitCode=0
Jul 16 20:41:52.514688 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:52.514203    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd" event={"ID":"babed4f2-aecb-4814-8da3-57fecb4617f6","Type":"ContainerDied","Data":"b9e0e342a7d954e609631a5c89e7a7bac801bfd50f18c9ffee7315a516cee57b"}
Jul 16 20:41:52.647515 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:52.647486    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd"
Jul 16 20:41:52.755777 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:52.755725    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/babed4f2-aecb-4814-8da3-57fecb4617f6-maas-api-service-ca\") pod \"babed4f2-aecb-4814-8da3-57fecb4617f6\" (UID: \"babed4f2-aecb-4814-8da3-57fecb4617f6\") "
Jul 16 20:41:52.756097 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:52.756074    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/babed4f2-aecb-4814-8da3-57fecb4617f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "babed4f2-aecb-4814-8da3-57fecb4617f6" (UID: "babed4f2-aecb-4814-8da3-57fecb4617f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:41:52.856925 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:52.856817    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/babed4f2-aecb-4814-8da3-57fecb4617f6-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:41:53.520899 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:53.520858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd" event={"ID":"babed4f2-aecb-4814-8da3-57fecb4617f6","Type":"ContainerDied","Data":"c406ffe06bbc58b5ed2cffc5b2f5d791baf0a6ca965e16ce49e8b650fbfdafdd"}
Jul 16 20:41:53.520899 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:53.520892    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd"
Jul 16 20:41:53.521358 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:41:53.520914    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c406ffe06bbc58b5ed2cffc5b2f5d791baf0a6ca965e16ce49e8b650fbfdafdd"
Jul 16 20:42:00.636214 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:00.636165    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"]
Jul 16 20:42:00.636706 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:00.636597    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d" podUID="f5ce85f5-6f1b-4466-815f-de6ceb5e9612" containerName="maas-api" containerID="cri-o://f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033" gracePeriod=30
Jul 16 20:42:00.890278 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:00.890217    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:42:00.933082 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:00.933047    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fss8v\" (UniqueName: \"kubernetes.io/projected/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-kube-api-access-fss8v\") pod \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") "
Jul 16 20:42:00.933275 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:00.933120    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-maas-api-tls\") pod \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") "
Jul 16 20:42:00.933275 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:00.933170    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-openshift-service-ca\") pod \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\" (UID: \"f5ce85f5-6f1b-4466-815f-de6ceb5e9612\") "
Jul 16 20:42:00.933645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:00.933603    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f5ce85f5-6f1b-4466-815f-de6ceb5e9612" (UID: "f5ce85f5-6f1b-4466-815f-de6ceb5e9612"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:42:00.935248 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:00.935228    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f5ce85f5-6f1b-4466-815f-de6ceb5e9612" (UID: "f5ce85f5-6f1b-4466-815f-de6ceb5e9612"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:42:00.935353 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:00.935331    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-kube-api-access-fss8v" (OuterVolumeSpecName: "kube-api-access-fss8v") pod "f5ce85f5-6f1b-4466-815f-de6ceb5e9612" (UID: "f5ce85f5-6f1b-4466-815f-de6ceb5e9612"). InnerVolumeSpecName "kube-api-access-fss8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:42:01.034104 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.034068    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fss8v\" (UniqueName: \"kubernetes.io/projected/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-kube-api-access-fss8v\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:42:01.034104 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.034098    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:42:01.034104 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.034108    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce85f5-6f1b-4466-815f-de6ceb5e9612-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:42:01.569300 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.569264    2579 generic.go:356] "Generic (PLEG): container finished" podID="f5ce85f5-6f1b-4466-815f-de6ceb5e9612" containerID="f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033" exitCode=0
Jul 16 20:42:01.569537 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.569312    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"
Jul 16 20:42:01.569537 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.569352    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d" event={"ID":"f5ce85f5-6f1b-4466-815f-de6ceb5e9612","Type":"ContainerDied","Data":"f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033"}
Jul 16 20:42:01.569537 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.569415    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d" event={"ID":"f5ce85f5-6f1b-4466-815f-de6ceb5e9612","Type":"ContainerDied","Data":"7ecc1861916aaf509ca5c4248f2284ffa3d01f9b3914e2c4585ff1523a389abe"}
Jul 16 20:42:01.569537 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.569461    2579 scope.go:117] "RemoveContainer" containerID="f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033"
Jul 16 20:42:01.580205 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.580190    2579 scope.go:117] "RemoveContainer" containerID="f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033"
Jul 16 20:42:01.580486 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:42:01.580464    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033\": container with ID starting with f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033 not found: ID does not exist" containerID="f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033"
Jul 16 20:42:01.580548 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.580493    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033"} err="failed to get container status \"f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033\": rpc error: code = NotFound desc = could not find container \"f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033\": container with ID starting with f3cfa57b805c76ee9e8afa944ffe82fe1941de89b7fb9489e5c7eef8342c8033 not found: ID does not exist"
Jul 16 20:42:01.592244 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.592214    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"]
Jul 16 20:42:01.594668 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:01.594648    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7cfc57-6d49db976c-7bc4d"]
Jul 16 20:42:02.410016 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:02.409981    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5ce85f5-6f1b-4466-815f-de6ceb5e9612" path="/var/lib/kubelet/pods/f5ce85f5-6f1b-4466-815f-de6ceb5e9612/volumes"
Jul 16 20:42:04.586609 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:04.586566    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:42:16.648935 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.648889    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c"]
Jul 16 20:42:16.649540 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.649500    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="babed4f2-aecb-4814-8da3-57fecb4617f6" containerName="revoke-keys"
Jul 16 20:42:16.649540 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.649523    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="babed4f2-aecb-4814-8da3-57fecb4617f6" containerName="revoke-keys"
Jul 16 20:42:16.649670 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.649555    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5ce85f5-6f1b-4466-815f-de6ceb5e9612" containerName="maas-api"
Jul 16 20:42:16.649670 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.649564    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5ce85f5-6f1b-4466-815f-de6ceb5e9612" containerName="maas-api"
Jul 16 20:42:16.649770 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.649701    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5ce85f5-6f1b-4466-815f-de6ceb5e9612" containerName="maas-api"
Jul 16 20:42:16.649770 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.649722    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="babed4f2-aecb-4814-8da3-57fecb4617f6" containerName="revoke-keys"
Jul 16 20:42:16.652959 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.652937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c"
Jul 16 20:42:16.655416 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.655391    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:42:16.665247 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.665222    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c"]
Jul 16 20:42:16.785894 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.785842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9128ec0-fd94-4962-88fc-6362710562a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1dad58-v976c\" (UID: \"c9128ec0-fd94-4962-88fc-6362710562a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c"
Jul 16 20:42:16.886833 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.886795    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9128ec0-fd94-4962-88fc-6362710562a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1dad58-v976c\" (UID: \"c9128ec0-fd94-4962-88fc-6362710562a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c"
Jul 16 20:42:16.887546 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.887524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9128ec0-fd94-4962-88fc-6362710562a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1dad58-v976c\" (UID: \"c9128ec0-fd94-4962-88fc-6362710562a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c"
Jul 16 20:42:16.964364 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:16.964331    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c"
Jul 16 20:42:17.114609 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:17.114578    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c"]
Jul 16 20:42:17.118252 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:42:17.118223    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9128ec0_fd94_4962_88fc_6362710562a9.slice/crio-d2b4b6b57fe0571d093866e8c95ee15395e6a18c8bc310fa52b1f4a61d7d4aa0 WatchSource:0}: Error finding container d2b4b6b57fe0571d093866e8c95ee15395e6a18c8bc310fa52b1f4a61d7d4aa0: Status 404 returned error can't find the container with id d2b4b6b57fe0571d093866e8c95ee15395e6a18c8bc310fa52b1f4a61d7d4aa0
Jul 16 20:42:17.665364 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:17.665326    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c" event={"ID":"c9128ec0-fd94-4962-88fc-6362710562a9","Type":"ContainerStarted","Data":"6894ca59aa5f0b350858be3576259d9e24e3a821b1493f772bd8c97010d2c425"}
Jul 16 20:42:17.665774 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:17.665392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c" event={"ID":"c9128ec0-fd94-4962-88fc-6362710562a9","Type":"ContainerStarted","Data":"d2b4b6b57fe0571d093866e8c95ee15395e6a18c8bc310fa52b1f4a61d7d4aa0"}
Jul 16 20:42:17.681036 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:17.680987    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c" podStartSLOduration=1.6809717659999999 podStartE2EDuration="1.680971766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:42:17.679730615 +0000 UTC m=+2501.827826036" watchObservedRunningTime="2026-07-16 20:42:17.680971766 +0000 UTC m=+2501.829067243"
Jul 16 20:42:18.672346 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:18.672305    2579 generic.go:356] "Generic (PLEG): container finished" podID="c9128ec0-fd94-4962-88fc-6362710562a9" containerID="6894ca59aa5f0b350858be3576259d9e24e3a821b1493f772bd8c97010d2c425" exitCode=0
Jul 16 20:42:18.672742 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:18.672352    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c" event={"ID":"c9128ec0-fd94-4962-88fc-6362710562a9","Type":"ContainerDied","Data":"6894ca59aa5f0b350858be3576259d9e24e3a821b1493f772bd8c97010d2c425"}
Jul 16 20:42:19.813324 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:19.813299    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c"
Jul 16 20:42:19.916391 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:19.916353    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9128ec0-fd94-4962-88fc-6362710562a9-maas-api-service-ca\") pod \"c9128ec0-fd94-4962-88fc-6362710562a9\" (UID: \"c9128ec0-fd94-4962-88fc-6362710562a9\") "
Jul 16 20:42:19.916772 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:19.916749    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9128ec0-fd94-4962-88fc-6362710562a9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c9128ec0-fd94-4962-88fc-6362710562a9" (UID: "c9128ec0-fd94-4962-88fc-6362710562a9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:42:20.017865 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:20.017778    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9128ec0-fd94-4962-88fc-6362710562a9-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:42:20.685001 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:20.684911    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c"
Jul 16 20:42:20.685001 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:20.684917    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1dad58-v976c" event={"ID":"c9128ec0-fd94-4962-88fc-6362710562a9","Type":"ContainerDied","Data":"d2b4b6b57fe0571d093866e8c95ee15395e6a18c8bc310fa52b1f4a61d7d4aa0"}
Jul 16 20:42:20.685202 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:20.685033    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2b4b6b57fe0571d093866e8c95ee15395e6a18c8bc310fa52b1f4a61d7d4aa0"
Jul 16 20:42:27.183200 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.183163    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"]
Jul 16 20:42:27.183738 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.183406    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5" podUID="97953106-7061-474f-99b3-304c6376a761" containerName="maas-api" containerID="cri-o://612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0" gracePeriod=30
Jul 16 20:42:27.435624 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.435555    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:42:27.588708 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.588677    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97953106-7061-474f-99b3-304c6376a761-maas-api-tls\") pod \"97953106-7061-474f-99b3-304c6376a761\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") "
Jul 16 20:42:27.588708 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.588711    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97953106-7061-474f-99b3-304c6376a761-openshift-service-ca\") pod \"97953106-7061-474f-99b3-304c6376a761\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") "
Jul 16 20:42:27.588944 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.588774    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv8jg\" (UniqueName: \"kubernetes.io/projected/97953106-7061-474f-99b3-304c6376a761-kube-api-access-pv8jg\") pod \"97953106-7061-474f-99b3-304c6376a761\" (UID: \"97953106-7061-474f-99b3-304c6376a761\") "
Jul 16 20:42:27.589123 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.589097    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97953106-7061-474f-99b3-304c6376a761-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "97953106-7061-474f-99b3-304c6376a761" (UID: "97953106-7061-474f-99b3-304c6376a761"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:42:27.590955 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.590921    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97953106-7061-474f-99b3-304c6376a761-kube-api-access-pv8jg" (OuterVolumeSpecName: "kube-api-access-pv8jg") pod "97953106-7061-474f-99b3-304c6376a761" (UID: "97953106-7061-474f-99b3-304c6376a761"). InnerVolumeSpecName "kube-api-access-pv8jg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:42:27.591065 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.590989    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97953106-7061-474f-99b3-304c6376a761-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "97953106-7061-474f-99b3-304c6376a761" (UID: "97953106-7061-474f-99b3-304c6376a761"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:42:27.690588 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.690498    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97953106-7061-474f-99b3-304c6376a761-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:42:27.690588 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.690539    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97953106-7061-474f-99b3-304c6376a761-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:42:27.690588 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.690550    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv8jg\" (UniqueName: \"kubernetes.io/projected/97953106-7061-474f-99b3-304c6376a761-kube-api-access-pv8jg\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:42:27.728280 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.728243    2579 generic.go:356] "Generic (PLEG): container finished" podID="97953106-7061-474f-99b3-304c6376a761" containerID="612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0" exitCode=0
Jul 16 20:42:27.728442 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.728293    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"
Jul 16 20:42:27.728442 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.728331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5" event={"ID":"97953106-7061-474f-99b3-304c6376a761","Type":"ContainerDied","Data":"612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0"}
Jul 16 20:42:27.728442 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.728386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5" event={"ID":"97953106-7061-474f-99b3-304c6376a761","Type":"ContainerDied","Data":"3f6021b7d7f2ce4b55a26d398d827e9afd0b2c5526110502924c10d794ec9ec9"}
Jul 16 20:42:27.728442 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.728410    2579 scope.go:117] "RemoveContainer" containerID="612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0"
Jul 16 20:42:27.740301 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.740285    2579 scope.go:117] "RemoveContainer" containerID="612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0"
Jul 16 20:42:27.740562 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:42:27.740541    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0\": container with ID starting with 612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0 not found: ID does not exist" containerID="612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0"
Jul 16 20:42:27.740640 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.740569    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0"} err="failed to get container status \"612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0\": rpc error: code = NotFound desc = could not find container \"612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0\": container with ID starting with 612c8e5f14437bebaa7fe915798871525cdf84ab2db9a4675e4bb928ac6ccdc0 not found: ID does not exist"
Jul 16 20:42:27.752410 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.752383    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"]
Jul 16 20:42:27.756054 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:27.756029    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1dad58-f787d7976-9t6p5"]
Jul 16 20:42:28.404213 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:28.404178    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97953106-7061-474f-99b3-304c6376a761" path="/var/lib/kubelet/pods/97953106-7061-474f-99b3-304c6376a761/volumes"
Jul 16 20:42:30.187036 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:30.186986    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:42:46.295845 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.295809    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"]
Jul 16 20:42:46.296472 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.296452    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9128ec0-fd94-4962-88fc-6362710562a9" containerName="revoke-keys"
Jul 16 20:42:46.296472 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.296474    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9128ec0-fd94-4962-88fc-6362710562a9" containerName="revoke-keys"
Jul 16 20:42:46.296645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.296487    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97953106-7061-474f-99b3-304c6376a761" containerName="maas-api"
Jul 16 20:42:46.296645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.296495    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="97953106-7061-474f-99b3-304c6376a761" containerName="maas-api"
Jul 16 20:42:46.296645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.296622    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9128ec0-fd94-4962-88fc-6362710562a9" containerName="revoke-keys"
Jul 16 20:42:46.296645 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.296638    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="97953106-7061-474f-99b3-304c6376a761" containerName="maas-api"
Jul 16 20:42:46.299612 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.299590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:46.302012 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.301985    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-9b1256-serving-cert\""
Jul 16 20:42:46.302124 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.301990    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 20:42:46.302338 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.302323    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g78x2\""
Jul 16 20:42:46.302789 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.302772    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 20:42:46.313701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.313676    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"]
Jul 16 20:42:46.357802 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.357774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2dst\" (UniqueName: \"kubernetes.io/projected/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-kube-api-access-f2dst\") pod \"maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:46.357925 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.357858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:46.357969 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.357921    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:46.459307 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.459270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:46.459536 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.459335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2dst\" (UniqueName: \"kubernetes.io/projected/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-kube-api-access-f2dst\") pod \"maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:46.459536 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.459369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:46.459536 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:42:46.459470    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-serving-cert: secret "maas-api-e2e-shared-a-9b1256-serving-cert" not found
Jul 16 20:42:46.459720 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:42:46.459540    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-maas-api-tls podName:8972ae04-7ccd-404f-a94f-2a1206c8b7fe nodeName:}" failed. No retries permitted until 2026-07-16 20:42:46.959519314 +0000 UTC m=+2531.107614721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-maas-api-tls") pod "maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk" (UID: "8972ae04-7ccd-404f-a94f-2a1206c8b7fe") : secret "maas-api-e2e-shared-a-9b1256-serving-cert" not found
Jul 16 20:42:46.460027 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.460009    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:46.471110 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.471078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2dst\" (UniqueName: \"kubernetes.io/projected/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-kube-api-access-f2dst\") pod \"maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:46.964115 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.964075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:46.966579 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:46.966554    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:47.210314 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:47.210270    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:47.344527 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:47.344503    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"]
Jul 16 20:42:47.346379 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:42:47.346344    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8972ae04_7ccd_404f_a94f_2a1206c8b7fe.slice/crio-d3a6e762e429bd485206044318b19bbdef3b111e443ad7b56031752bb92671e5 WatchSource:0}: Error finding container d3a6e762e429bd485206044318b19bbdef3b111e443ad7b56031752bb92671e5: Status 404 returned error can't find the container with id d3a6e762e429bd485206044318b19bbdef3b111e443ad7b56031752bb92671e5
Jul 16 20:42:47.852887 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:47.852808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk" event={"ID":"8972ae04-7ccd-404f-a94f-2a1206c8b7fe","Type":"ContainerStarted","Data":"d3a6e762e429bd485206044318b19bbdef3b111e443ad7b56031752bb92671e5"}
Jul 16 20:42:48.860260 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:48.860213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk" event={"ID":"8972ae04-7ccd-404f-a94f-2a1206c8b7fe","Type":"ContainerStarted","Data":"753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e"}
Jul 16 20:42:48.860671 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:48.860353    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:48.878622 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:48.878572    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk" podStartSLOduration=2.3844441769999998 podStartE2EDuration="2.878556255s" totalImagesPullingTime="494.112078ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:42:48.876499415 +0000 UTC m=+2533.024594838" watchObservedRunningTime="2026-07-16 20:42:48.878556255 +0000 UTC m=+2533.026651729"
Jul 16 20:42:50.987799 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:50.987749    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:42:54.392395 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.392357    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"]
Jul 16 20:42:54.396047 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.396026    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:54.398164 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.398142    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-8a0b5e-serving-cert\""
Jul 16 20:42:54.407545 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.407519    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"]
Jul 16 20:42:54.541891 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.541849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621c0302-ac74-48ae-b8ca-8e436f9ec568-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:54.542065 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.541907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621c0302-ac74-48ae-b8ca-8e436f9ec568-maas-api-tls\") pod \"maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:54.542065 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.541962    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqsx4\" (UniqueName: \"kubernetes.io/projected/621c0302-ac74-48ae-b8ca-8e436f9ec568-kube-api-access-dqsx4\") pod \"maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:54.642905 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.642817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621c0302-ac74-48ae-b8ca-8e436f9ec568-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:54.642905 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.642868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621c0302-ac74-48ae-b8ca-8e436f9ec568-maas-api-tls\") pod \"maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:54.642905 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.642898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqsx4\" (UniqueName: \"kubernetes.io/projected/621c0302-ac74-48ae-b8ca-8e436f9ec568-kube-api-access-dqsx4\") pod \"maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:54.643160 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:42:54.642998    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-serving-cert: secret "maas-api-e2e-shared-b-8a0b5e-serving-cert" not found
Jul 16 20:42:54.643160 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:42:54.643060    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/621c0302-ac74-48ae-b8ca-8e436f9ec568-maas-api-tls podName:621c0302-ac74-48ae-b8ca-8e436f9ec568 nodeName:}" failed. No retries permitted until 2026-07-16 20:42:55.143043687 +0000 UTC m=+2539.291139092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/621c0302-ac74-48ae-b8ca-8e436f9ec568-maas-api-tls") pod "maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g" (UID: "621c0302-ac74-48ae-b8ca-8e436f9ec568") : secret "maas-api-e2e-shared-b-8a0b5e-serving-cert" not found
Jul 16 20:42:54.643501 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.643482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621c0302-ac74-48ae-b8ca-8e436f9ec568-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:54.651050 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.651019    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqsx4\" (UniqueName: \"kubernetes.io/projected/621c0302-ac74-48ae-b8ca-8e436f9ec568-kube-api-access-dqsx4\") pod \"maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:54.872288 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:54.872259    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:42:55.147591 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:55.147550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621c0302-ac74-48ae-b8ca-8e436f9ec568-maas-api-tls\") pod \"maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:55.150129 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:55.150103    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621c0302-ac74-48ae-b8ca-8e436f9ec568-maas-api-tls\") pod \"maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:55.307568 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:55.307531    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:55.439111 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:55.439088    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"]
Jul 16 20:42:55.440509 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:42:55.440482    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621c0302_ac74_48ae_b8ca_8e436f9ec568.slice/crio-e8aadafdf1c639bbe166011ddbe893b2b0a906e57e15276d68f0a97ef45f75fb WatchSource:0}: Error finding container e8aadafdf1c639bbe166011ddbe893b2b0a906e57e15276d68f0a97ef45f75fb: Status 404 returned error can't find the container with id e8aadafdf1c639bbe166011ddbe893b2b0a906e57e15276d68f0a97ef45f75fb
Jul 16 20:42:55.902008 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:55.901969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g" event={"ID":"621c0302-ac74-48ae-b8ca-8e436f9ec568","Type":"ContainerStarted","Data":"e2b6fa99d889358666f39ca0f0e20030221a8c3e79408c94517b6ec18877bbdd"}
Jul 16 20:42:55.902161 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:55.902029    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g" event={"ID":"621c0302-ac74-48ae-b8ca-8e436f9ec568","Type":"ContainerStarted","Data":"e8aadafdf1c639bbe166011ddbe893b2b0a906e57e15276d68f0a97ef45f75fb"}
Jul 16 20:42:55.902161 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:55.902140    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:42:55.919666 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:42:55.919619    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g" podStartSLOduration=1.5846758969999999 podStartE2EDuration="1.919604081s" totalImagesPullingTime="334.928184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:42:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:42:55.91772657 +0000 UTC m=+2540.065821990" watchObservedRunningTime="2026-07-16 20:42:55.919604081 +0000 UTC m=+2540.067699503"
Jul 16 20:43:01.913060 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:01.913026    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:43:02.083293 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:02.083260    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:43:18.088627 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:18.088582    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:43:58.731794 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.731714    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"]
Jul 16 20:43:58.735040 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.735015    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:58.738100 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.738067    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8a0b5e\"/\"kube-root-ca.crt\""
Jul 16 20:43:58.738255 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.738072    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8a0b5e\"/\"default-dockercfg-kbrhp\""
Jul 16 20:43:58.738255 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.738072    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8a0b5e\"/\"auth-test-model-8a0b5e-kserve-self-signed-certs\""
Jul 16 20:43:58.738255 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.738169    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8a0b5e\"/\"openshift-service-ca.crt\""
Jul 16 20:43:58.745491 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.745458    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"]
Jul 16 20:43:58.914539 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.914497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-model-cache\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:58.914748 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.914588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-home\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:58.914748 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.914627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-dshm\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:58.914748 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.914642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tmp-dir\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:58.914748 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.914661    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tls-certs\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:58.914938 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:58.914755    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84rwr\" (UniqueName: \"kubernetes.io/projected/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-kube-api-access-84rwr\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.015852 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.015758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-home\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.015852 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.015813    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-dshm\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.015852 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.015838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tmp-dir\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.016164 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.015862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tls-certs\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.016164 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.015920    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84rwr\" (UniqueName: \"kubernetes.io/projected/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-kube-api-access-84rwr\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.016164 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.015958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-model-cache\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.016328 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.016256    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tmp-dir\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.016328 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.016311    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-home\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.016415 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.016327    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-model-cache\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.018236 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.018211    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-dshm\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.018391 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.018370    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tls-certs\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.024276 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.024254    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84rwr\" (UniqueName: \"kubernetes.io/projected/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-kube-api-access-84rwr\") pod \"auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") " pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.046856 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.046826    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:43:59.185033 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.185004    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"]
Jul 16 20:43:59.187020 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:43:59.186975    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cb17803_fa4a_41d3_8451_633d5a3a3cd7.slice/crio-ffff27d85ade8f30f5474db6f65d5feeb5a22fc47c7e777c096718f735b365e0 WatchSource:0}: Error finding container ffff27d85ade8f30f5474db6f65d5feeb5a22fc47c7e777c096718f735b365e0: Status 404 returned error can't find the container with id ffff27d85ade8f30f5474db6f65d5feeb5a22fc47c7e777c096718f735b365e0
Jul 16 20:43:59.261065 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.261009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx" event={"ID":"8cb17803-fa4a-41d3-8451-633d5a3a3cd7","Type":"ContainerStarted","Data":"82515aebe35ac3daaacc046a207798cb20ff074111ea264411390057c08961f3"}
Jul 16 20:43:59.261268 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.261076    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx" event={"ID":"8cb17803-fa4a-41d3-8451-633d5a3a3cd7","Type":"ContainerStarted","Data":"ffff27d85ade8f30f5474db6f65d5feeb5a22fc47c7e777c096718f735b365e0"}
Jul 16 20:43:59.280233 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:43:59.280124    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx" podStartSLOduration=1.280106849 podStartE2EDuration="1.280106849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:43:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:43:59.276904282 +0000 UTC m=+2603.424999716" watchObservedRunningTime="2026-07-16 20:43:59.280106849 +0000 UTC m=+2603.428202249"
Jul 16 20:44:02.378971 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:44:02.378928    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:44:09.047554 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:44:09.047516    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:44:09.048033 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:44:09.047571    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:44:09.060273 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:44:09.060241    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:44:09.334298 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:44:09.334207    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:44:17.218924 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:44:17.218888    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:44:32.076771 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:44:32.076723    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:44:46.607248 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:44:46.607218    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:45:00.133157 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.133118    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l"]
Jul 16 20:45:00.137124 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.137102    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737245-x8qkj"]
Jul 16 20:45:00.137258 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.137234    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l"
Jul 16 20:45:00.139481 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.139454    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:45:00.140332 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.140313    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l"]
Jul 16 20:45:00.140464 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.140411    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-x8qkj"
Jul 16 20:45:00.144365 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.144342    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l"
Jul 16 20:45:00.144664 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.144647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l"]
Jul 16 20:45:00.149217 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.149190    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737245-x8qkj"]
Jul 16 20:45:00.153488 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.153456    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l"
Jul 16 20:45:00.154732 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.154206    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l"]
Jul 16 20:45:00.162873 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.162854    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-x8qkj"
Jul 16 20:45:00.208022 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.207992    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l"
Jul 16 20:45:00.324811 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.324771    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l"]
Jul 16 20:45:00.325658 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:45:00.325607    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab8fb85_dd25_4e5a_be74_0dcb3bfbcef9.slice/crio-dff683b67b7fbec92eace804352fe7842eaf2ab8ca12bd3adf4c33857f0a3c1a WatchSource:0}: Error finding container dff683b67b7fbec92eace804352fe7842eaf2ab8ca12bd3adf4c33857f0a3c1a: Status 404 returned error can't find the container with id dff683b67b7fbec92eace804352fe7842eaf2ab8ca12bd3adf4c33857f0a3c1a
Jul 16 20:45:00.349285 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.349255    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737245-x8qkj"]
Jul 16 20:45:00.351492 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:45:00.351465    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8086a675_e415_4eca_b47d_0f46bf749efe.slice/crio-241303e1d7932ae2b41dcfcb8c006c2fc579a471f8201d6fa6dae82c08025715 WatchSource:0}: Error finding container 241303e1d7932ae2b41dcfcb8c006c2fc579a471f8201d6fa6dae82c08025715: Status 404 returned error can't find the container with id 241303e1d7932ae2b41dcfcb8c006c2fc579a471f8201d6fa6dae82c08025715
Jul 16 20:45:00.379020 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.378992    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l"]
Jul 16 20:45:00.387208 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:45:00.386336    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod808f2eda_0f89_4cf5_abe5_5ac8c7187011.slice/crio-6136ed904149d346f13dc0373478f6e92b4011c82a8edc253a9d18c48df4a342 WatchSource:0}: Error finding container 6136ed904149d346f13dc0373478f6e92b4011c82a8edc253a9d18c48df4a342: Status 404 returned error can't find the container with id 6136ed904149d346f13dc0373478f6e92b4011c82a8edc253a9d18c48df4a342
Jul 16 20:45:00.616664 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.616634    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l" event={"ID":"808f2eda-0f89-4cf5-abe5-5ac8c7187011","Type":"ContainerStarted","Data":"6136ed904149d346f13dc0373478f6e92b4011c82a8edc253a9d18c48df4a342"}
Jul 16 20:45:00.617999 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.617968    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l" event={"ID":"3ab8fb85-dd25-4e5a-be74-0dcb3bfbcef9","Type":"ContainerStarted","Data":"dff683b67b7fbec92eace804352fe7842eaf2ab8ca12bd3adf4c33857f0a3c1a"}
Jul 16 20:45:00.619413 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:00.619391    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-x8qkj" event={"ID":"8086a675-e415-4eca-b47d-0f46bf749efe","Type":"ContainerStarted","Data":"241303e1d7932ae2b41dcfcb8c006c2fc579a471f8201d6fa6dae82c08025715"}
Jul 16 20:45:01.627412 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:01.627328    2579 generic.go:356] "Generic (PLEG): container finished" podID="808f2eda-0f89-4cf5-abe5-5ac8c7187011" containerID="8c4b2a463415aae57de7f5a5c4bb6299d537ec55c203d12cb3727d0443974f1c" exitCode=0
Jul 16 20:45:01.627893 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:01.627414    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l" event={"ID":"808f2eda-0f89-4cf5-abe5-5ac8c7187011","Type":"ContainerDied","Data":"8c4b2a463415aae57de7f5a5c4bb6299d537ec55c203d12cb3727d0443974f1c"}
Jul 16 20:45:01.628915 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:01.628891    2579 generic.go:356] "Generic (PLEG): container finished" podID="3ab8fb85-dd25-4e5a-be74-0dcb3bfbcef9" containerID="c3e810b0f0fda05b228843a0b7ad7610a72983248d775ea6dbf7c666c6ef35d1" exitCode=0
Jul 16 20:45:01.629025 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:01.628970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l" event={"ID":"3ab8fb85-dd25-4e5a-be74-0dcb3bfbcef9","Type":"ContainerDied","Data":"c3e810b0f0fda05b228843a0b7ad7610a72983248d775ea6dbf7c666c6ef35d1"}
Jul 16 20:45:01.630451 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:01.630415    2579 generic.go:356] "Generic (PLEG): container finished" podID="8086a675-e415-4eca-b47d-0f46bf749efe" containerID="15606908c2d2ef77f377674ca1c858b6baa3e721db3224054a6b92ca662c1005" exitCode=0
Jul 16 20:45:01.630551 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:01.630471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-x8qkj" event={"ID":"8086a675-e415-4eca-b47d-0f46bf749efe","Type":"ContainerDied","Data":"15606908c2d2ef77f377674ca1c858b6baa3e721db3224054a6b92ca662c1005"}
Jul 16 20:45:02.587572 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:02.587528    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:45:02.784387 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:02.784356    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-x8qkj"
Jul 16 20:45:02.836989 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:02.836966    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l"
Jul 16 20:45:02.840103 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:02.840045    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l"
Jul 16 20:45:03.643653 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:03.643624    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l"
Jul 16 20:45:03.643857 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:03.643666    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l" event={"ID":"808f2eda-0f89-4cf5-abe5-5ac8c7187011","Type":"ContainerDied","Data":"6136ed904149d346f13dc0373478f6e92b4011c82a8edc253a9d18c48df4a342"}
Jul 16 20:45:03.643857 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:03.643711    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6136ed904149d346f13dc0373478f6e92b4011c82a8edc253a9d18c48df4a342"
Jul 16 20:45:03.644997 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:03.644970    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l"
Jul 16 20:45:03.645140 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:03.645002    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l" event={"ID":"3ab8fb85-dd25-4e5a-be74-0dcb3bfbcef9","Type":"ContainerDied","Data":"dff683b67b7fbec92eace804352fe7842eaf2ab8ca12bd3adf4c33857f0a3c1a"}
Jul 16 20:45:03.645140 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:03.645042    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dff683b67b7fbec92eace804352fe7842eaf2ab8ca12bd3adf4c33857f0a3c1a"
Jul 16 20:45:03.646219 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:03.646200    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-x8qkj" event={"ID":"8086a675-e415-4eca-b47d-0f46bf749efe","Type":"ContainerDied","Data":"241303e1d7932ae2b41dcfcb8c006c2fc579a471f8201d6fa6dae82c08025715"}
Jul 16 20:45:03.646219 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:03.646214    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737245-x8qkj"
Jul 16 20:45:03.646357 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:03.646233    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="241303e1d7932ae2b41dcfcb8c006c2fc579a471f8201d6fa6dae82c08025715"
Jul 16 20:45:21.791254 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:21.791214    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:45:40.793178 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:40.793142    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:45:59.683898 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:45:59.683862    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:46:03.925234 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:03.925200    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"]
Jul 16 20:46:17.290532 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:17.290494    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:46:18.945705 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:18.945640    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx" podUID="8cb17803-fa4a-41d3-8451-633d5a3a3cd7" containerName="main" containerID="cri-o://82515aebe35ac3daaacc046a207798cb20ff074111ea264411390057c08961f3" gracePeriod=45
Jul 16 20:46:19.104617 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.104579    2579 generic.go:356] "Generic (PLEG): container finished" podID="8cb17803-fa4a-41d3-8451-633d5a3a3cd7" containerID="82515aebe35ac3daaacc046a207798cb20ff074111ea264411390057c08961f3" exitCode=0
Jul 16 20:46:19.104769 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.104648    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx" event={"ID":"8cb17803-fa4a-41d3-8451-633d5a3a3cd7","Type":"ContainerDied","Data":"82515aebe35ac3daaacc046a207798cb20ff074111ea264411390057c08961f3"}
Jul 16 20:46:19.195098 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.195074    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:46:19.295380 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295298    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-home\") pod \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") "
Jul 16 20:46:19.295380 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295342    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tls-certs\") pod \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") "
Jul 16 20:46:19.295620 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295464    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-model-cache\") pod \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") "
Jul 16 20:46:19.295620 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295500    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tmp-dir\") pod \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") "
Jul 16 20:46:19.295620 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295551    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-dshm\") pod \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") "
Jul 16 20:46:19.295620 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295579    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84rwr\" (UniqueName: \"kubernetes.io/projected/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-kube-api-access-84rwr\") pod \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\" (UID: \"8cb17803-fa4a-41d3-8451-633d5a3a3cd7\") "
Jul 16 20:46:19.295620 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295594    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-home" (OuterVolumeSpecName: "home") pod "8cb17803-fa4a-41d3-8451-633d5a3a3cd7" (UID: "8cb17803-fa4a-41d3-8451-633d5a3a3cd7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:46:19.295876 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295733    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8cb17803-fa4a-41d3-8451-633d5a3a3cd7" (UID: "8cb17803-fa4a-41d3-8451-633d5a3a3cd7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:46:19.295876 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295759    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-model-cache" (OuterVolumeSpecName: "model-cache") pod "8cb17803-fa4a-41d3-8451-633d5a3a3cd7" (UID: "8cb17803-fa4a-41d3-8451-633d5a3a3cd7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:46:19.295990 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295914    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-home\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:46:19.295990 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295927    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-model-cache\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:46:19.295990 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.295938    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tmp-dir\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:46:19.297793 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.297756    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-kube-api-access-84rwr" (OuterVolumeSpecName: "kube-api-access-84rwr") pod "8cb17803-fa4a-41d3-8451-633d5a3a3cd7" (UID: "8cb17803-fa4a-41d3-8451-633d5a3a3cd7"). InnerVolumeSpecName "kube-api-access-84rwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:46:19.297915 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.297804    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-dshm" (OuterVolumeSpecName: "dshm") pod "8cb17803-fa4a-41d3-8451-633d5a3a3cd7" (UID: "8cb17803-fa4a-41d3-8451-633d5a3a3cd7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:46:19.297915 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.297821    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8cb17803-fa4a-41d3-8451-633d5a3a3cd7" (UID: "8cb17803-fa4a-41d3-8451-633d5a3a3cd7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:46:19.396355 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.396327    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-tls-certs\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:46:19.396355 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.396352    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-dshm\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:46:19.396528 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:19.396363    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84rwr\" (UniqueName: \"kubernetes.io/projected/8cb17803-fa4a-41d3-8451-633d5a3a3cd7-kube-api-access-84rwr\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:46:20.112709 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:20.112679    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"
Jul 16 20:46:20.113185 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:20.112709    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx" event={"ID":"8cb17803-fa4a-41d3-8451-633d5a3a3cd7","Type":"ContainerDied","Data":"ffff27d85ade8f30f5474db6f65d5feeb5a22fc47c7e777c096718f735b365e0"}
Jul 16 20:46:20.113185 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:20.112768    2579 scope.go:117] "RemoveContainer" containerID="82515aebe35ac3daaacc046a207798cb20ff074111ea264411390057c08961f3"
Jul 16 20:46:20.136774 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:20.136720    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"]
Jul 16 20:46:20.139813 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:20.139791    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/auth-test-model-8a0b5e-kserve-5b5b8f5cf5-cwmtx"]
Jul 16 20:46:20.405450 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:20.405348    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cb17803-fa4a-41d3-8451-633d5a3a3cd7" path="/var/lib/kubelet/pods/8cb17803-fa4a-41d3-8451-633d5a3a3cd7/volumes"
Jul 16 20:46:29.790585 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:46:29.790553    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:47:14.438004 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.437970    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"]
Jul 16 20:47:14.438502 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438365    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb17803-fa4a-41d3-8451-633d5a3a3cd7" containerName="main"
Jul 16 20:47:14.438502 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438381    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb17803-fa4a-41d3-8451-633d5a3a3cd7" containerName="main"
Jul 16 20:47:14.438502 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438408    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8086a675-e415-4eca-b47d-0f46bf749efe" containerName="cleanup"
Jul 16 20:47:14.438502 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438442    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8086a675-e415-4eca-b47d-0f46bf749efe" containerName="cleanup"
Jul 16 20:47:14.438502 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438452    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ab8fb85-dd25-4e5a-be74-0dcb3bfbcef9" containerName="cleanup"
Jul 16 20:47:14.438502 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438462    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ab8fb85-dd25-4e5a-be74-0dcb3bfbcef9" containerName="cleanup"
Jul 16 20:47:14.438502 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438472    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="808f2eda-0f89-4cf5-abe5-5ac8c7187011" containerName="cleanup"
Jul 16 20:47:14.438502 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438477    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="808f2eda-0f89-4cf5-abe5-5ac8c7187011" containerName="cleanup"
Jul 16 20:47:14.438773 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438542    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ab8fb85-dd25-4e5a-be74-0dcb3bfbcef9" containerName="cleanup"
Jul 16 20:47:14.438773 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438554    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb17803-fa4a-41d3-8451-633d5a3a3cd7" containerName="main"
Jul 16 20:47:14.438773 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438563    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="808f2eda-0f89-4cf5-abe5-5ac8c7187011" containerName="cleanup"
Jul 16 20:47:14.438773 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.438569    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8086a675-e415-4eca-b47d-0f46bf749efe" containerName="cleanup"
Jul 16 20:47:14.441561 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.441544    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.444582 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.444558    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8a0b5e\"/\"kube-root-ca.crt\""
Jul 16 20:47:14.444582 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.444567    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8a0b5e\"/\"sub-test-model-8a0b5e-kserve-self-signed-certs\""
Jul 16 20:47:14.444772 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.444558    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8a0b5e\"/\"default-dockercfg-kbrhp\""
Jul 16 20:47:14.444772 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.444595    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8a0b5e\"/\"openshift-service-ca.crt\""
Jul 16 20:47:14.450387 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.450360    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"]
Jul 16 20:47:14.571863 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.571836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-home\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.572027 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.571891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6khr\" (UniqueName: \"kubernetes.io/projected/6b144290-bcba-4877-8582-4470121d304f-kube-api-access-r6khr\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.572027 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.571945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-dshm\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.572027 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.571970    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b144290-bcba-4877-8582-4470121d304f-tls-certs\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.572177 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.572103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-tmp-dir\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.572177 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.572149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-model-cache\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.673151 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.673112    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-home\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.673339 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.673162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6khr\" (UniqueName: \"kubernetes.io/projected/6b144290-bcba-4877-8582-4470121d304f-kube-api-access-r6khr\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.673339 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.673193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-dshm\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.673339 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.673211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b144290-bcba-4877-8582-4470121d304f-tls-certs\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.673339 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.673233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-tmp-dir\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.673339 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.673259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-model-cache\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.673684 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.673604    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-home\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.673743 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.673675    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-tmp-dir\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.673743 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.673710    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-model-cache\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.675500 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.675473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-dshm\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.675820 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.675803    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b144290-bcba-4877-8582-4470121d304f-tls-certs\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.680675 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.680648    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6khr\" (UniqueName: \"kubernetes.io/projected/6b144290-bcba-4877-8582-4470121d304f-kube-api-access-r6khr\") pod \"sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") " pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.751827 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.751728    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:14.888450 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:14.888411    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"]
Jul 16 20:47:14.890039 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:47:14.890013    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b144290_bcba_4877_8582_4470121d304f.slice/crio-99a6dc45a47c643e2372fd598bb91886a9cbeb37e1fe4017a3c2bfe04742d18c WatchSource:0}: Error finding container 99a6dc45a47c643e2372fd598bb91886a9cbeb37e1fe4017a3c2bfe04742d18c: Status 404 returned error can't find the container with id 99a6dc45a47c643e2372fd598bb91886a9cbeb37e1fe4017a3c2bfe04742d18c
Jul 16 20:47:15.438392 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:15.438356    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j" event={"ID":"6b144290-bcba-4877-8582-4470121d304f","Type":"ContainerStarted","Data":"9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49"}
Jul 16 20:47:15.438806 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:15.438402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j" event={"ID":"6b144290-bcba-4877-8582-4470121d304f","Type":"ContainerStarted","Data":"99a6dc45a47c643e2372fd598bb91886a9cbeb37e1fe4017a3c2bfe04742d18c"}
Jul 16 20:47:15.457896 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:15.457840    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j" podStartSLOduration=1.457816698 podStartE2EDuration="1.457816698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:47:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:47:15.454589623 +0000 UTC m=+2799.602685067" watchObservedRunningTime="2026-07-16 20:47:15.457816698 +0000 UTC m=+2799.605912120"
Jul 16 20:47:18.588287 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:18.588245    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:47:24.752349 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:24.752310    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:24.752843 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:24.752363    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:24.765228 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:24.765202    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:25.508011 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:25.507981    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:47:34.281101 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:34.281065    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:47:52.623699 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:52.623667    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"]
Jul 16 20:47:56.386315 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:47:56.386273    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:48:07.647885 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.647840    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j" podUID="6b144290-bcba-4877-8582-4470121d304f" containerName="main" containerID="cri-o://9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49" gracePeriod=45
Jul 16 20:48:07.898884 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.898822    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:48:07.943978 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.943949    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6khr\" (UniqueName: \"kubernetes.io/projected/6b144290-bcba-4877-8582-4470121d304f-kube-api-access-r6khr\") pod \"6b144290-bcba-4877-8582-4470121d304f\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") "
Jul 16 20:48:07.943978 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.943983    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-dshm\") pod \"6b144290-bcba-4877-8582-4470121d304f\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") "
Jul 16 20:48:07.944224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.944009    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b144290-bcba-4877-8582-4470121d304f-tls-certs\") pod \"6b144290-bcba-4877-8582-4470121d304f\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") "
Jul 16 20:48:07.944224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.944049    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-model-cache\") pod \"6b144290-bcba-4877-8582-4470121d304f\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") "
Jul 16 20:48:07.944224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.944067    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-tmp-dir\") pod \"6b144290-bcba-4877-8582-4470121d304f\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") "
Jul 16 20:48:07.944224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.944094    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-home\") pod \"6b144290-bcba-4877-8582-4470121d304f\" (UID: \"6b144290-bcba-4877-8582-4470121d304f\") "
Jul 16 20:48:07.944461 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.944356    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b144290-bcba-4877-8582-4470121d304f" (UID: "6b144290-bcba-4877-8582-4470121d304f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:48:07.944461 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.944367    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b144290-bcba-4877-8582-4470121d304f" (UID: "6b144290-bcba-4877-8582-4470121d304f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:48:07.944592 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.944497    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-home" (OuterVolumeSpecName: "home") pod "6b144290-bcba-4877-8582-4470121d304f" (UID: "6b144290-bcba-4877-8582-4470121d304f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:48:07.944592 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.944516    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-model-cache\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:48:07.944592 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.944536    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-tmp-dir\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:48:07.946191 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.946161    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-dshm" (OuterVolumeSpecName: "dshm") pod "6b144290-bcba-4877-8582-4470121d304f" (UID: "6b144290-bcba-4877-8582-4470121d304f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:48:07.946298 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.946200    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b144290-bcba-4877-8582-4470121d304f-kube-api-access-r6khr" (OuterVolumeSpecName: "kube-api-access-r6khr") pod "6b144290-bcba-4877-8582-4470121d304f" (UID: "6b144290-bcba-4877-8582-4470121d304f"). InnerVolumeSpecName "kube-api-access-r6khr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:48:07.946298 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:07.946208    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b144290-bcba-4877-8582-4470121d304f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b144290-bcba-4877-8582-4470121d304f" (UID: "6b144290-bcba-4877-8582-4470121d304f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:48:08.045616 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.045583    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6khr\" (UniqueName: \"kubernetes.io/projected/6b144290-bcba-4877-8582-4470121d304f-kube-api-access-r6khr\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:48:08.045616 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.045609    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-dshm\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:48:08.045616 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.045619    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b144290-bcba-4877-8582-4470121d304f-tls-certs\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:48:08.045835 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.045628    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b144290-bcba-4877-8582-4470121d304f-home\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:48:08.755561 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.755474    2579 generic.go:356] "Generic (PLEG): container finished" podID="6b144290-bcba-4877-8582-4470121d304f" containerID="9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49" exitCode=0
Jul 16 20:48:08.755561 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.755518    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"
Jul 16 20:48:08.756022 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.755562    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j" event={"ID":"6b144290-bcba-4877-8582-4470121d304f","Type":"ContainerDied","Data":"9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49"}
Jul 16 20:48:08.756022 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.755607    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j" event={"ID":"6b144290-bcba-4877-8582-4470121d304f","Type":"ContainerDied","Data":"99a6dc45a47c643e2372fd598bb91886a9cbeb37e1fe4017a3c2bfe04742d18c"}
Jul 16 20:48:08.756022 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.755626    2579 scope.go:117] "RemoveContainer" containerID="9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49"
Jul 16 20:48:08.764930 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.764911    2579 scope.go:117] "RemoveContainer" containerID="9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49"
Jul 16 20:48:08.765189 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:48:08.765171    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49\": container with ID starting with 9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49 not found: ID does not exist" containerID="9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49"
Jul 16 20:48:08.765234 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.765197    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49"} err="failed to get container status \"9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49\": rpc error: code = NotFound desc = could not find container \"9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49\": container with ID starting with 9df9485a52f758968fdb81fbce76089b5a6766588e780734928ba77d7f3c9c49 not found: ID does not exist"
Jul 16 20:48:08.772973 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.772948    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"]
Jul 16 20:48:08.775893 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:08.775873    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8a0b5e/sub-test-model-8a0b5e-kserve-66c4fb9dcc-gq64j"]
Jul 16 20:48:10.404281 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:10.404248    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b144290-bcba-4877-8582-4470121d304f" path="/var/lib/kubelet/pods/6b144290-bcba-4877-8582-4470121d304f/volumes"
Jul 16 20:48:11.098140 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:11.098108    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:48:25.181659 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:25.181617    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:48:51.302240 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:48:51.302205    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:49:05.793903 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:05.793867    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:49:15.398619 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.398581    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7357-139849676282800"]
Jul 16 20:49:15.399164 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.399141    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b144290-bcba-4877-8582-4470121d304f" containerName="main"
Jul 16 20:49:15.399258 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.399166    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b144290-bcba-4877-8582-4470121d304f" containerName="main"
Jul 16 20:49:15.399316 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.399307    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b144290-bcba-4877-8582-4470121d304f" containerName="main"
Jul 16 20:49:15.409735 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.409706    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7357-139849676282800"
Jul 16 20:49:15.410365 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.410317    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7357-139849676282800"]
Jul 16 20:49:15.412355 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.412334    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lhm6n\""
Jul 16 20:49:15.507164 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.507137    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rql7p\" (UniqueName: \"kubernetes.io/projected/903d157e-548f-49e1-b330-800c5ce5f537-kube-api-access-rql7p\") pod \"test-curl-7357-139849676282800\" (UID: \"903d157e-548f-49e1-b330-800c5ce5f537\") " pod="opendatahub/test-curl-7357-139849676282800"
Jul 16 20:49:15.607777 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.607745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rql7p\" (UniqueName: \"kubernetes.io/projected/903d157e-548f-49e1-b330-800c5ce5f537-kube-api-access-rql7p\") pod \"test-curl-7357-139849676282800\" (UID: \"903d157e-548f-49e1-b330-800c5ce5f537\") " pod="opendatahub/test-curl-7357-139849676282800"
Jul 16 20:49:15.615421 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.615395    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rql7p\" (UniqueName: \"kubernetes.io/projected/903d157e-548f-49e1-b330-800c5ce5f537-kube-api-access-rql7p\") pod \"test-curl-7357-139849676282800\" (UID: \"903d157e-548f-49e1-b330-800c5ce5f537\") " pod="opendatahub/test-curl-7357-139849676282800"
Jul 16 20:49:15.719867 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.719818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7357-139849676282800"
Jul 16 20:49:15.851022 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.850998    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7357-139849676282800"]
Jul 16 20:49:15.852881 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:49:15.852849    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod903d157e_548f_49e1_b330_800c5ce5f537.slice/crio-cfe91405ae804e5ae191b67d741ad56a6de9f408ee554f3b06d84da011b16e7c WatchSource:0}: Error finding container cfe91405ae804e5ae191b67d741ad56a6de9f408ee554f3b06d84da011b16e7c: Status 404 returned error can't find the container with id cfe91405ae804e5ae191b67d741ad56a6de9f408ee554f3b06d84da011b16e7c
Jul 16 20:49:15.854577 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:15.854558    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 20:49:16.153531 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:16.153410    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7357-139849676282800" event={"ID":"903d157e-548f-49e1-b330-800c5ce5f537","Type":"ContainerStarted","Data":"cfe91405ae804e5ae191b67d741ad56a6de9f408ee554f3b06d84da011b16e7c"}
Jul 16 20:49:17.160906 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:17.160873    2579 generic.go:356] "Generic (PLEG): container finished" podID="903d157e-548f-49e1-b330-800c5ce5f537" containerID="d33df83c843ce00c0071fd00e6b0f4f8c3f0c9b7891147b3e98c99b8a95fbb4a" exitCode=0
Jul 16 20:49:17.161281 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:17.160973    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7357-139849676282800" event={"ID":"903d157e-548f-49e1-b330-800c5ce5f537","Type":"ContainerDied","Data":"d33df83c843ce00c0071fd00e6b0f4f8c3f0c9b7891147b3e98c99b8a95fbb4a"}
Jul 16 20:49:18.302001 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:18.301970    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7357-139849676282800"
Jul 16 20:49:18.331478 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:18.331448    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rql7p\" (UniqueName: \"kubernetes.io/projected/903d157e-548f-49e1-b330-800c5ce5f537-kube-api-access-rql7p\") pod \"903d157e-548f-49e1-b330-800c5ce5f537\" (UID: \"903d157e-548f-49e1-b330-800c5ce5f537\") "
Jul 16 20:49:18.333594 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:18.333566    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/903d157e-548f-49e1-b330-800c5ce5f537-kube-api-access-rql7p" (OuterVolumeSpecName: "kube-api-access-rql7p") pod "903d157e-548f-49e1-b330-800c5ce5f537" (UID: "903d157e-548f-49e1-b330-800c5ce5f537"). InnerVolumeSpecName "kube-api-access-rql7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:49:18.337024 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:18.336992    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7357-139849676282800_903d157e-548f-49e1-b330-800c5ce5f537/test-curl-7357-139849676282800/0.log"
Jul 16 20:49:18.364037 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:18.364015    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7357-139849676282800"]
Jul 16 20:49:18.366552 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:18.366533    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7357-139849676282800"]
Jul 16 20:49:18.404735 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:18.404705    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="903d157e-548f-49e1-b330-800c5ce5f537" path="/var/lib/kubelet/pods/903d157e-548f-49e1-b330-800c5ce5f537/volumes"
Jul 16 20:49:18.433282 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:18.433204    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rql7p\" (UniqueName: \"kubernetes.io/projected/903d157e-548f-49e1-b330-800c5ce5f537-kube-api-access-rql7p\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:49:19.174531 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:19.174499    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7357-139849676282800"
Jul 16 20:49:19.174687 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:19.174535    2579 scope.go:117] "RemoveContainer" containerID="d33df83c843ce00c0071fd00e6b0f4f8c3f0c9b7891147b3e98c99b8a95fbb4a"
Jul 16 20:49:19.495923 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:19.495838    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:49:33.409011 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.408969    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w"]
Jul 16 20:49:33.409392 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.409373    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="903d157e-548f-49e1-b330-800c5ce5f537" containerName="test-curl-7357-139849676282800"
Jul 16 20:49:33.409392 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.409386    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="903d157e-548f-49e1-b330-800c5ce5f537" containerName="test-curl-7357-139849676282800"
Jul 16 20:49:33.409524 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.409459    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="903d157e-548f-49e1-b330-800c5ce5f537" containerName="test-curl-7357-139849676282800"
Jul 16 20:49:33.413834 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.413816    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w"
Jul 16 20:49:33.416311 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.416290    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:49:33.422020 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.421998    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w"]
Jul 16 20:49:33.476423 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.476389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f16cfc4f-c5d2-4ed3-b545-dddb89001d31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w\" (UID: \"f16cfc4f-c5d2-4ed3-b545-dddb89001d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w"
Jul 16 20:49:33.577643 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.577613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f16cfc4f-c5d2-4ed3-b545-dddb89001d31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w\" (UID: \"f16cfc4f-c5d2-4ed3-b545-dddb89001d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w"
Jul 16 20:49:33.578213 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.578192    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f16cfc4f-c5d2-4ed3-b545-dddb89001d31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w\" (UID: \"f16cfc4f-c5d2-4ed3-b545-dddb89001d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w"
Jul 16 20:49:33.725521 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.725491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w"
Jul 16 20:49:33.861548 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:33.861520    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w"]
Jul 16 20:49:33.862411 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:49:33.862388    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16cfc4f_c5d2_4ed3_b545_dddb89001d31.slice/crio-2cf17d103119aca9f3285a15eabdbf81c2bc18f98feb5207add43cd7867defcc WatchSource:0}: Error finding container 2cf17d103119aca9f3285a15eabdbf81c2bc18f98feb5207add43cd7867defcc: Status 404 returned error can't find the container with id 2cf17d103119aca9f3285a15eabdbf81c2bc18f98feb5207add43cd7867defcc
Jul 16 20:49:34.263667 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:34.263635    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w" event={"ID":"f16cfc4f-c5d2-4ed3-b545-dddb89001d31","Type":"ContainerStarted","Data":"597f08a259f6954abc4c445d19746cebad73ee7672ffa0d1e8c73ffbb1b51fc0"}
Jul 16 20:49:34.263800 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:34.263692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w" event={"ID":"f16cfc4f-c5d2-4ed3-b545-dddb89001d31","Type":"ContainerStarted","Data":"2cf17d103119aca9f3285a15eabdbf81c2bc18f98feb5207add43cd7867defcc"}
Jul 16 20:49:34.279922 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:34.279873    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w" podStartSLOduration=1.279857572 podStartE2EDuration="1.279857572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:49:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:49:34.279508862 +0000 UTC m=+2938.427604286" watchObservedRunningTime="2026-07-16 20:49:34.279857572 +0000 UTC m=+2938.427952995"
Jul 16 20:49:35.271497 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:35.271462    2579 generic.go:356] "Generic (PLEG): container finished" podID="f16cfc4f-c5d2-4ed3-b545-dddb89001d31" containerID="597f08a259f6954abc4c445d19746cebad73ee7672ffa0d1e8c73ffbb1b51fc0" exitCode=0
Jul 16 20:49:35.271888 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:35.271531    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w" event={"ID":"f16cfc4f-c5d2-4ed3-b545-dddb89001d31","Type":"ContainerDied","Data":"597f08a259f6954abc4c445d19746cebad73ee7672ffa0d1e8c73ffbb1b51fc0"}
Jul 16 20:49:36.409652 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:36.409629    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w"
Jul 16 20:49:36.507752 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:36.507722    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f16cfc4f-c5d2-4ed3-b545-dddb89001d31-maas-api-service-ca\") pod \"f16cfc4f-c5d2-4ed3-b545-dddb89001d31\" (UID: \"f16cfc4f-c5d2-4ed3-b545-dddb89001d31\") "
Jul 16 20:49:36.508073 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:36.508050    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16cfc4f-c5d2-4ed3-b545-dddb89001d31-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f16cfc4f-c5d2-4ed3-b545-dddb89001d31" (UID: "f16cfc4f-c5d2-4ed3-b545-dddb89001d31"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:49:36.609303 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:36.609236    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f16cfc4f-c5d2-4ed3-b545-dddb89001d31-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:49:37.283879 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:37.283849    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w"
Jul 16 20:49:37.284067 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:37.283897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w" event={"ID":"f16cfc4f-c5d2-4ed3-b545-dddb89001d31","Type":"ContainerDied","Data":"2cf17d103119aca9f3285a15eabdbf81c2bc18f98feb5207add43cd7867defcc"}
Jul 16 20:49:37.284067 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:37.283950    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2cf17d103119aca9f3285a15eabdbf81c2bc18f98feb5207add43cd7867defcc"
Jul 16 20:49:44.072115 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.072071    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"]
Jul 16 20:49:44.072505 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.072323    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk" podUID="8972ae04-7ccd-404f-a94f-2a1206c8b7fe" containerName="maas-api" containerID="cri-o://753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e" gracePeriod=30
Jul 16 20:49:44.104726 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.104697    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l"]
Jul 16 20:49:44.110554 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.110525    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b1256-29737245-mc22l"]
Jul 16 20:49:44.318490 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.318465    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:49:44.329552 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.329490    2579 generic.go:356] "Generic (PLEG): container finished" podID="8972ae04-7ccd-404f-a94f-2a1206c8b7fe" containerID="753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e" exitCode=0
Jul 16 20:49:44.329552 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.329530    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"
Jul 16 20:49:44.329712 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.329550    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk" event={"ID":"8972ae04-7ccd-404f-a94f-2a1206c8b7fe","Type":"ContainerDied","Data":"753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e"}
Jul 16 20:49:44.329712 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.329634    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk" event={"ID":"8972ae04-7ccd-404f-a94f-2a1206c8b7fe","Type":"ContainerDied","Data":"d3a6e762e429bd485206044318b19bbdef3b111e443ad7b56031752bb92671e5"}
Jul 16 20:49:44.329712 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.329656    2579 scope.go:117] "RemoveContainer" containerID="753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e"
Jul 16 20:49:44.339815 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.339797    2579 scope.go:117] "RemoveContainer" containerID="753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e"
Jul 16 20:49:44.340102 ip-10-0-140-236 kubenswrapper[2579]: E0716 20:49:44.340082    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e\": container with ID starting with 753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e not found: ID does not exist" containerID="753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e"
Jul 16 20:49:44.340165 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.340113    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e"} err="failed to get container status \"753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e\": rpc error: code = NotFound desc = could not find container \"753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e\": container with ID starting with 753a71956a115d76d74221bb27e65ab1cca0247d4e763bd510e00c09f614513e not found: ID does not exist"
Jul 16 20:49:44.403667 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.403634    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="808f2eda-0f89-4cf5-abe5-5ac8c7187011" path="/var/lib/kubelet/pods/808f2eda-0f89-4cf5-abe5-5ac8c7187011/volumes"
Jul 16 20:49:44.478818 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.478789    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2dst\" (UniqueName: \"kubernetes.io/projected/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-kube-api-access-f2dst\") pod \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") "
Jul 16 20:49:44.479004 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.478850    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-openshift-service-ca\") pod \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") "
Jul 16 20:49:44.479004 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.478908    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-maas-api-tls\") pod \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\" (UID: \"8972ae04-7ccd-404f-a94f-2a1206c8b7fe\") "
Jul 16 20:49:44.479229 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.479197    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8972ae04-7ccd-404f-a94f-2a1206c8b7fe" (UID: "8972ae04-7ccd-404f-a94f-2a1206c8b7fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:49:44.479403 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.479383    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:49:44.481008 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.480978    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8972ae04-7ccd-404f-a94f-2a1206c8b7fe" (UID: "8972ae04-7ccd-404f-a94f-2a1206c8b7fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:49:44.481084 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.481070    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-kube-api-access-f2dst" (OuterVolumeSpecName: "kube-api-access-f2dst") pod "8972ae04-7ccd-404f-a94f-2a1206c8b7fe" (UID: "8972ae04-7ccd-404f-a94f-2a1206c8b7fe"). InnerVolumeSpecName "kube-api-access-f2dst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:49:44.579852 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.579781    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2dst\" (UniqueName: \"kubernetes.io/projected/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-kube-api-access-f2dst\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:49:44.579852 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.579808    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8972ae04-7ccd-404f-a94f-2a1206c8b7fe-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:49:44.659612 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.659581    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"]
Jul 16 20:49:44.666543 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:44.666520    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b1256-6b97c55b76-68jtk"]
Jul 16 20:49:46.404120 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:46.404089    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8972ae04-7ccd-404f-a94f-2a1206c8b7fe" path="/var/lib/kubelet/pods/8972ae04-7ccd-404f-a94f-2a1206c8b7fe/volumes"
Jul 16 20:49:47.587469 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:47.587422    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:49:59.649937 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.649856    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn"]
Jul 16 20:49:59.650301 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.650227    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8972ae04-7ccd-404f-a94f-2a1206c8b7fe" containerName="maas-api"
Jul 16 20:49:59.650301 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.650238    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8972ae04-7ccd-404f-a94f-2a1206c8b7fe" containerName="maas-api"
Jul 16 20:49:59.650301 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.650262    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16cfc4f-c5d2-4ed3-b545-dddb89001d31" containerName="revoke-keys"
Jul 16 20:49:59.650301 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.650267    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16cfc4f-c5d2-4ed3-b545-dddb89001d31" containerName="revoke-keys"
Jul 16 20:49:59.650456 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.650325    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8972ae04-7ccd-404f-a94f-2a1206c8b7fe" containerName="maas-api"
Jul 16 20:49:59.650456 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.650333    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16cfc4f-c5d2-4ed3-b545-dddb89001d31" containerName="revoke-keys"
Jul 16 20:49:59.655425 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.655403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn"
Jul 16 20:49:59.658069 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.658048    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fsnh7\""
Jul 16 20:49:59.663096 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.663077    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn"]
Jul 16 20:49:59.710303 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.710270    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24aa29c8-216a-4603-84b9-7cd34be874e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn\" (UID: \"24aa29c8-216a-4603-84b9-7cd34be874e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn"
Jul 16 20:49:59.811496 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.811458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24aa29c8-216a-4603-84b9-7cd34be874e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn\" (UID: \"24aa29c8-216a-4603-84b9-7cd34be874e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn"
Jul 16 20:49:59.812122 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.812097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24aa29c8-216a-4603-84b9-7cd34be874e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn\" (UID: \"24aa29c8-216a-4603-84b9-7cd34be874e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn"
Jul 16 20:49:59.966572 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:49:59.966535    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn"
Jul 16 20:50:00.107126 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:00.107096    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn"]
Jul 16 20:50:00.108541 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:50:00.108510    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24aa29c8_216a_4603_84b9_7cd34be874e5.slice/crio-cc05e638f3afccd72ceab88d264caa7035447b1ce71b1ccdd5c79708b3c5b750 WatchSource:0}: Error finding container cc05e638f3afccd72ceab88d264caa7035447b1ce71b1ccdd5c79708b3c5b750: Status 404 returned error can't find the container with id cc05e638f3afccd72ceab88d264caa7035447b1ce71b1ccdd5c79708b3c5b750
Jul 16 20:50:00.432961 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:00.432920    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn" event={"ID":"24aa29c8-216a-4603-84b9-7cd34be874e5","Type":"ContainerStarted","Data":"1a2168ced90eee2bc0d9d8b95efd688696db304a0d3620d894cdf8059ae7d239"}
Jul 16 20:50:00.433170 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:00.433003    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn" event={"ID":"24aa29c8-216a-4603-84b9-7cd34be874e5","Type":"ContainerStarted","Data":"cc05e638f3afccd72ceab88d264caa7035447b1ce71b1ccdd5c79708b3c5b750"}
Jul 16 20:50:00.450747 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:00.450698    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn" podStartSLOduration=1.450686073 podStartE2EDuration="1.450686073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:50:00.44763814 +0000 UTC m=+2964.595733563" watchObservedRunningTime="2026-07-16 20:50:00.450686073 +0000 UTC m=+2964.598781550"
Jul 16 20:50:01.440688 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:01.440651    2579 generic.go:356] "Generic (PLEG): container finished" podID="24aa29c8-216a-4603-84b9-7cd34be874e5" containerID="1a2168ced90eee2bc0d9d8b95efd688696db304a0d3620d894cdf8059ae7d239" exitCode=0
Jul 16 20:50:01.441094 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:01.440739    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn" event={"ID":"24aa29c8-216a-4603-84b9-7cd34be874e5","Type":"ContainerDied","Data":"1a2168ced90eee2bc0d9d8b95efd688696db304a0d3620d894cdf8059ae7d239"}
Jul 16 20:50:02.578384 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:02.578357    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn"
Jul 16 20:50:02.631610 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:02.631587    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24aa29c8-216a-4603-84b9-7cd34be874e5-maas-api-service-ca\") pod \"24aa29c8-216a-4603-84b9-7cd34be874e5\" (UID: \"24aa29c8-216a-4603-84b9-7cd34be874e5\") "
Jul 16 20:50:02.631945 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:02.631926    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24aa29c8-216a-4603-84b9-7cd34be874e5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "24aa29c8-216a-4603-84b9-7cd34be874e5" (UID: "24aa29c8-216a-4603-84b9-7cd34be874e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:50:02.732572 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:02.732510    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24aa29c8-216a-4603-84b9-7cd34be874e5-maas-api-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:50:03.454175 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:03.454143    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn"
Jul 16 20:50:03.454355 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:03.454182    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn" event={"ID":"24aa29c8-216a-4603-84b9-7cd34be874e5","Type":"ContainerDied","Data":"cc05e638f3afccd72ceab88d264caa7035447b1ce71b1ccdd5c79708b3c5b750"}
Jul 16 20:50:03.454355 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:03.454233    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc05e638f3afccd72ceab88d264caa7035447b1ce71b1ccdd5c79708b3c5b750"
Jul 16 20:50:10.298902 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.298867    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"]
Jul 16 20:50:10.299424 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.299065    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g" podUID="621c0302-ac74-48ae-b8ca-8e436f9ec568" containerName="maas-api" containerID="cri-o://e2b6fa99d889358666f39ca0f0e20030221a8c3e79408c94517b6ec18877bbdd" gracePeriod=30
Jul 16 20:50:10.333663 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.333623    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l"]
Jul 16 20:50:10.336007 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.335984    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8a0b5e-29737245-btz8l"]
Jul 16 20:50:10.404739 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.404708    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ab8fb85-dd25-4e5a-be74-0dcb3bfbcef9" path="/var/lib/kubelet/pods/3ab8fb85-dd25-4e5a-be74-0dcb3bfbcef9/volumes"
Jul 16 20:50:10.502179 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.502144    2579 generic.go:356] "Generic (PLEG): container finished" podID="621c0302-ac74-48ae-b8ca-8e436f9ec568" containerID="e2b6fa99d889358666f39ca0f0e20030221a8c3e79408c94517b6ec18877bbdd" exitCode=0
Jul 16 20:50:10.502338 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.502201    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g" event={"ID":"621c0302-ac74-48ae-b8ca-8e436f9ec568","Type":"ContainerDied","Data":"e2b6fa99d889358666f39ca0f0e20030221a8c3e79408c94517b6ec18877bbdd"}
Jul 16 20:50:10.549817 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.549751    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:50:10.595938 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.595896    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqsx4\" (UniqueName: \"kubernetes.io/projected/621c0302-ac74-48ae-b8ca-8e436f9ec568-kube-api-access-dqsx4\") pod \"621c0302-ac74-48ae-b8ca-8e436f9ec568\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") "
Jul 16 20:50:10.596140 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.595953    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621c0302-ac74-48ae-b8ca-8e436f9ec568-openshift-service-ca\") pod \"621c0302-ac74-48ae-b8ca-8e436f9ec568\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") "
Jul 16 20:50:10.596140 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.595993    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621c0302-ac74-48ae-b8ca-8e436f9ec568-maas-api-tls\") pod \"621c0302-ac74-48ae-b8ca-8e436f9ec568\" (UID: \"621c0302-ac74-48ae-b8ca-8e436f9ec568\") "
Jul 16 20:50:10.596300 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.596275    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/621c0302-ac74-48ae-b8ca-8e436f9ec568-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "621c0302-ac74-48ae-b8ca-8e436f9ec568" (UID: "621c0302-ac74-48ae-b8ca-8e436f9ec568"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:50:10.598176 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.598148    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/621c0302-ac74-48ae-b8ca-8e436f9ec568-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "621c0302-ac74-48ae-b8ca-8e436f9ec568" (UID: "621c0302-ac74-48ae-b8ca-8e436f9ec568"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:50:10.598176 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.598153    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621c0302-ac74-48ae-b8ca-8e436f9ec568-kube-api-access-dqsx4" (OuterVolumeSpecName: "kube-api-access-dqsx4") pod "621c0302-ac74-48ae-b8ca-8e436f9ec568" (UID: "621c0302-ac74-48ae-b8ca-8e436f9ec568"). InnerVolumeSpecName "kube-api-access-dqsx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:50:10.697714 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.697678    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqsx4\" (UniqueName: \"kubernetes.io/projected/621c0302-ac74-48ae-b8ca-8e436f9ec568-kube-api-access-dqsx4\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:50:10.697714 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.697710    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621c0302-ac74-48ae-b8ca-8e436f9ec568-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:50:10.697714 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:10.697720    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621c0302-ac74-48ae-b8ca-8e436f9ec568-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\""
Jul 16 20:50:11.509638 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:11.509599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g" event={"ID":"621c0302-ac74-48ae-b8ca-8e436f9ec568","Type":"ContainerDied","Data":"e8aadafdf1c639bbe166011ddbe893b2b0a906e57e15276d68f0a97ef45f75fb"}
Jul 16 20:50:11.510084 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:11.509660    2579 scope.go:117] "RemoveContainer" containerID="e2b6fa99d889358666f39ca0f0e20030221a8c3e79408c94517b6ec18877bbdd"
Jul 16 20:50:11.510084 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:11.509619    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"
Jul 16 20:50:11.533478 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:11.533452    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"]
Jul 16 20:50:11.537401 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:11.537378    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8a0b5e-fcc55945f-qrr9g"]
Jul 16 20:50:12.404904 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:12.404872    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="621c0302-ac74-48ae-b8ca-8e436f9ec568" path="/var/lib/kubelet/pods/621c0302-ac74-48ae-b8ca-8e436f9ec568/volumes"
Jul 16 20:50:13.197940 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:13.197908    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9x54h"]
Jul 16 20:50:32.160570 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:32.160539    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64c594d8d4-9vdc5_dd439004-1886-4e2e-bdac-ee5c110a0a07/copy-manifests/0.log"
Jul 16 20:50:32.170040 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:32.170002    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64c594d8d4-9vdc5_dd439004-1886-4e2e-bdac-ee5c110a0a07/manager/0.log"
Jul 16 20:50:33.030757 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:33.030732    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737230-7qf2d_d577682e-3e76-44c7-b0ac-cf0580cdd2d2/cleanup/0.log"
Jul 16 20:50:33.146815 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:33.146784    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737245-x8qkj_8086a675-e415-4eca-b47d-0f46bf749efe/cleanup/0.log"
Jul 16 20:50:33.250729 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:33.250699    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-49aaf8ce-qckpd_772ee840-6fdb-48be-9648-3f6ac86aa33b/revoke-keys/0.log"
Jul 16 20:50:33.361993 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:33.361906    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-e9ceb054-kqqwt_249c3330-2d24-4e47-ac22-31a48f8a0ea7/revoke-keys/0.log"
Jul 16 20:50:33.501538 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:33.501507    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-7cfc57-k9txd_babed4f2-aecb-4814-8da3-57fecb4617f6/revoke-keys/0.log"
Jul 16 20:50:33.620675 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:33.620577    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-1dad58-v976c_c9128ec0-fd94-4962-88fc-6362710562a9/revoke-keys/0.log"
Jul 16 20:50:33.752304 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:33.752264    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-0d61d6f5-4hd89_fd1e3f8d-b7aa-465e-ad72-021624cd6cd5/revoke-keys/0.log"
Jul 16 20:50:33.862292 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:33.862262    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-955113-spn59_e8bdb14d-54bd-4219-838e-eba2f815bc21/revoke-keys/0.log"
Jul 16 20:50:33.980832 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:33.980803    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-e69c2e-fv424_fdb73860-9f1a-4c9d-8546-be9ec64feb55/revoke-keys/0.log"
Jul 16 20:50:34.104844 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:34.104804    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-fa642852-rzjrx_ccb7ba31-08d4-4c86-a5a0-a7209ac1e65d/revoke-keys/0.log"
Jul 16 20:50:34.220701 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:34.220677    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-087d0429-r48nl_d29186dd-9ad3-474f-b4e8-763027b7e7ee/revoke-keys/0.log"
Jul 16 20:50:34.336885 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:34.336803    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-0faadf3f-54qdb_6a260ab4-4a87-491e-81ad-893e08424d43/revoke-keys/0.log"
Jul 16 20:50:34.444271 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:34.444239    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-9b1256-q8l5w_f16cfc4f-c5d2-4ed3-b545-dddb89001d31/revoke-keys/0.log"
Jul 16 20:50:34.554399 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:34.554354    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-8a0b5e-ftjwn_24aa29c8-216a-4603-84b9-7cd34be874e5/revoke-keys/0.log"
Jul 16 20:50:35.449159 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.449128    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45_48e55e72-0725-434d-bd51-c5bb9a0e64a2/util/0.log"
Jul 16 20:50:35.455372 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.455341    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45_48e55e72-0725-434d-bd51-c5bb9a0e64a2/pull/0.log"
Jul 16 20:50:35.461092 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.461061    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45_48e55e72-0725-434d-bd51-c5bb9a0e64a2/extract/0.log"
Jul 16 20:50:35.573178 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.573150    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs_ee7a12b4-7f78-47c6-a283-9a706cdad756/util/0.log"
Jul 16 20:50:35.580191 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.580170    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs_ee7a12b4-7f78-47c6-a283-9a706cdad756/pull/0.log"
Jul 16 20:50:35.586492 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.586473    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs_ee7a12b4-7f78-47c6-a283-9a706cdad756/extract/0.log"
Jul 16 20:50:35.698240 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.698201    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx_b448c7f8-6112-4834-adad-30534db365e9/util/0.log"
Jul 16 20:50:35.706409 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.706335    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx_b448c7f8-6112-4834-adad-30534db365e9/pull/0.log"
Jul 16 20:50:35.714908 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.714888    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx_b448c7f8-6112-4834-adad-30534db365e9/extract/0.log"
Jul 16 20:50:35.828220 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.828194    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw_1758a2d6-6453-47f2-b2aa-f947a054bf68/util/0.log"
Jul 16 20:50:35.834636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.834606    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw_1758a2d6-6453-47f2-b2aa-f947a054bf68/pull/0.log"
Jul 16 20:50:35.840990 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:35.840963    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw_1758a2d6-6453-47f2-b2aa-f947a054bf68/extract/0.log"
Jul 16 20:50:36.702107 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:36.702074    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-9x54h_04363917-f191-4630-85b3-3532f8019d61/limitador/0.log"
Jul 16 20:50:37.344932 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:37.344892    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7rz45_c2eea5a7-b748-4e9a-91a2-10a6b82d87b9/discovery/0.log"
Jul 16 20:50:37.959365 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:37.959331    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nsfxk_a192e8a0-baa2-4783-a097-bb862e2859a2/payload-processing/0.log"
Jul 16 20:50:38.676946 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:38.676916    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-9v4f9_9b79b743-e531-47dc-a537-7bd99093365b/main/0.log"
Jul 16 20:50:47.919159 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:47.919108    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j665j_381ae42e-c8c0-4763-b7c1-ce7fef4d6f64/global-pull-secret-syncer/0.log"
Jul 16 20:50:48.061910 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:48.061877    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cs26l_6f39627a-2f22-4b29-91b8-1b871181fe77/konnectivity-agent/0.log"
Jul 16 20:50:48.180313 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:48.180226    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-236.ec2.internal_b2bd58dc38956b1c8ecf2cf823b81fbf/haproxy/0.log"
Jul 16 20:50:52.450078 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.450044    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45_48e55e72-0725-434d-bd51-c5bb9a0e64a2/extract/0.log"
Jul 16 20:50:52.473303 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.473255    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45_48e55e72-0725-434d-bd51-c5bb9a0e64a2/util/0.log"
Jul 16 20:50:52.498539 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.498506    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595nt45_48e55e72-0725-434d-bd51-c5bb9a0e64a2/pull/0.log"
Jul 16 20:50:52.530918 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.530892    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs_ee7a12b4-7f78-47c6-a283-9a706cdad756/extract/0.log"
Jul 16 20:50:52.558767 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.558737    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs_ee7a12b4-7f78-47c6-a283-9a706cdad756/util/0.log"
Jul 16 20:50:52.589203 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.589178    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qldrs_ee7a12b4-7f78-47c6-a283-9a706cdad756/pull/0.log"
Jul 16 20:50:52.627007 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.626979    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx_b448c7f8-6112-4834-adad-30534db365e9/extract/0.log"
Jul 16 20:50:52.649030 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.649006    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx_b448c7f8-6112-4834-adad-30534db365e9/util/0.log"
Jul 16 20:50:52.676304 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.676281    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8krx_b448c7f8-6112-4834-adad-30534db365e9/pull/0.log"
Jul 16 20:50:52.708560 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.708489    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw_1758a2d6-6453-47f2-b2aa-f947a054bf68/extract/0.log"
Jul 16 20:50:52.752406 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.752383    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw_1758a2d6-6453-47f2-b2aa-f947a054bf68/util/0.log"
Jul 16 20:50:52.798367 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:52.798337    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xtscw_1758a2d6-6453-47f2-b2aa-f947a054bf68/pull/0.log"
Jul 16 20:50:53.833565 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:53.833528    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-9x54h_04363917-f191-4630-85b3-3532f8019d61/limitador/0.log"
Jul 16 20:50:55.516951 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.516911    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4227dd95-01d9-4cfd-9a7b-e0e1e22a824d/alertmanager/0.log"
Jul 16 20:50:55.570019 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.569990    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4227dd95-01d9-4cfd-9a7b-e0e1e22a824d/config-reloader/0.log"
Jul 16 20:50:55.644487 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.644461    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4227dd95-01d9-4cfd-9a7b-e0e1e22a824d/kube-rbac-proxy-web/0.log"
Jul 16 20:50:55.687636 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.687612    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4227dd95-01d9-4cfd-9a7b-e0e1e22a824d/kube-rbac-proxy/0.log"
Jul 16 20:50:55.711190 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.711168    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4227dd95-01d9-4cfd-9a7b-e0e1e22a824d/kube-rbac-proxy-metric/0.log"
Jul 16 20:50:55.738216 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.738194    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4227dd95-01d9-4cfd-9a7b-e0e1e22a824d/prom-label-proxy/0.log"
Jul 16 20:50:55.764536 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.764513    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4227dd95-01d9-4cfd-9a7b-e0e1e22a824d/init-config-reloader/0.log"
Jul 16 20:50:55.860282 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.860241    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-2q6ts_32996e00-92c4-447f-936e-b3ddbbfd50e1/kube-state-metrics/0.log"
Jul 16 20:50:55.878567 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.878538    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-2q6ts_32996e00-92c4-447f-936e-b3ddbbfd50e1/kube-rbac-proxy-main/0.log"
Jul 16 20:50:55.899768 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.899739    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-2q6ts_32996e00-92c4-447f-936e-b3ddbbfd50e1/kube-rbac-proxy-self/0.log"
Jul 16 20:50:55.927301 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.927273    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-749644bcf7-f6frz_e0972057-5af7-41b5-9ac4-ca57a836167a/metrics-server/0.log"
Jul 16 20:50:55.950930 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.950904    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-8vkv4_e0157262-e344-4390-a1cf-df0028852271/monitoring-plugin/0.log"
Jul 16 20:50:55.979919 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.979899    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lw4q4_d459df04-b13a-455d-8c30-e190c2478496/node-exporter/0.log"
Jul 16 20:50:55.999663 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:55.999637    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lw4q4_d459df04-b13a-455d-8c30-e190c2478496/kube-rbac-proxy/0.log"
Jul 16 20:50:56.023918 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.023878    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lw4q4_d459df04-b13a-455d-8c30-e190c2478496/init-textfile/0.log"
Jul 16 20:50:56.242738 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.242705    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-p2khk_c3471f9a-d501-4ead-a27b-372dbdfd1e18/kube-rbac-proxy-main/0.log"
Jul 16 20:50:56.264609 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.264581    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-p2khk_c3471f9a-d501-4ead-a27b-372dbdfd1e18/kube-rbac-proxy-self/0.log"
Jul 16 20:50:56.285541 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.285511    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-p2khk_c3471f9a-d501-4ead-a27b-372dbdfd1e18/openshift-state-metrics/0.log"
Jul 16 20:50:56.585452 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.585329    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56c4c79f8d-kw55p_fef3b439-79a8-4245-90dc-968f429abe7d/telemeter-client/0.log"
Jul 16 20:50:56.606476 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.606450    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56c4c79f8d-kw55p_fef3b439-79a8-4245-90dc-968f429abe7d/reload/0.log"
Jul 16 20:50:56.629607 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.629579    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56c4c79f8d-kw55p_fef3b439-79a8-4245-90dc-968f429abe7d/kube-rbac-proxy/0.log"
Jul 16 20:50:56.670664 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.670637    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764c4bdfd-zhln4_aa9b886a-6291-4502-ac97-d140b1853806/thanos-query/0.log"
Jul 16 20:50:56.695060 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.695020    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764c4bdfd-zhln4_aa9b886a-6291-4502-ac97-d140b1853806/kube-rbac-proxy-web/0.log"
Jul 16 20:50:56.721699 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.721676    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764c4bdfd-zhln4_aa9b886a-6291-4502-ac97-d140b1853806/kube-rbac-proxy/0.log"
Jul 16 20:50:56.768403 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.768376    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764c4bdfd-zhln4_aa9b886a-6291-4502-ac97-d140b1853806/prom-label-proxy/0.log"
Jul 16 20:50:56.819381 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.819361    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764c4bdfd-zhln4_aa9b886a-6291-4502-ac97-d140b1853806/kube-rbac-proxy-rules/0.log"
Jul 16 20:50:56.907540 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:56.907448    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-764c4bdfd-zhln4_aa9b886a-6291-4502-ac97-d140b1853806/kube-rbac-proxy-metrics/0.log"
Jul 16 20:50:58.405858 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.405822    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"]
Jul 16 20:50:58.406224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.406162    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621c0302-ac74-48ae-b8ca-8e436f9ec568" containerName="maas-api"
Jul 16 20:50:58.406224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.406173    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="621c0302-ac74-48ae-b8ca-8e436f9ec568" containerName="maas-api"
Jul 16 20:50:58.406224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.406193    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24aa29c8-216a-4603-84b9-7cd34be874e5" containerName="revoke-keys"
Jul 16 20:50:58.406224 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.406199    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="24aa29c8-216a-4603-84b9-7cd34be874e5" containerName="revoke-keys"
Jul 16 20:50:58.406358 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.406264    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="24aa29c8-216a-4603-84b9-7cd34be874e5" containerName="revoke-keys"
Jul 16 20:50:58.406358 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.406274    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="621c0302-ac74-48ae-b8ca-8e436f9ec568" containerName="maas-api"
Jul 16 20:50:58.409534 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.409512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.411532 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.411509    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lw5rh\"/\"openshift-service-ca.crt\""
Jul 16 20:50:58.411698 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.411680    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lw5rh\"/\"kube-root-ca.crt\""
Jul 16 20:50:58.412244 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.412228    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lw5rh\"/\"default-dockercfg-xznfh\""
Jul 16 20:50:58.421379 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.421346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"]
Jul 16 20:50:58.522034 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.522008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-proc\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.522034 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.522036    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntgxl\" (UniqueName: \"kubernetes.io/projected/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-kube-api-access-ntgxl\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.522209 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.522060    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-podres\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.522209 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.522197    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-sys\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.522277 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.522222    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-lib-modules\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.623552 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.623519    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-sys\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.623552 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.623553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-lib-modules\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.623737 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.623592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-proc\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.623737 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.623616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntgxl\" (UniqueName: \"kubernetes.io/projected/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-kube-api-access-ntgxl\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.623737 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.623644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-sys\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.623737 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.623655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-podres\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.623737 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.623705    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-lib-modules\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.623737 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.623720    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-proc\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.624027 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.623740    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-podres\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.635279 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.635251    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntgxl\" (UniqueName: \"kubernetes.io/projected/51e59847-cfdc-40be-a7a5-e3d4feaf3bae-kube-api-access-ntgxl\") pod \"perf-node-gather-daemonset-t8x5j\" (UID: \"51e59847-cfdc-40be-a7a5-e3d4feaf3bae\") " pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.719881 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.719854    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:58.851667 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:58.851635    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"]
Jul 16 20:50:58.852928 ip-10-0-140-236 kubenswrapper[2579]: W0716 20:50:58.852899    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod51e59847_cfdc_40be_a7a5_e3d4feaf3bae.slice/crio-0df601d6a62ee54e508bb1c5a3c0866f5f4d41597ed23948e743dc59772fce22 WatchSource:0}: Error finding container 0df601d6a62ee54e508bb1c5a3c0866f5f4d41597ed23948e743dc59772fce22: Status 404 returned error can't find the container with id 0df601d6a62ee54e508bb1c5a3c0866f5f4d41597ed23948e743dc59772fce22
Jul 16 20:50:59.187638 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:59.187610    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b786d75f6-27729_2f84a6f0-8a18-4f41-8586-27cf56f70ec0/console/0.log"
Jul 16 20:50:59.803305 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:59.803263    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j" event={"ID":"51e59847-cfdc-40be-a7a5-e3d4feaf3bae","Type":"ContainerStarted","Data":"718f08d10851027d2a45d891daede19471af5ca8cd3226d8e6cbeb5b4f803d74"}
Jul 16 20:50:59.803717 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:59.803323    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j" event={"ID":"51e59847-cfdc-40be-a7a5-e3d4feaf3bae","Type":"ContainerStarted","Data":"0df601d6a62ee54e508bb1c5a3c0866f5f4d41597ed23948e743dc59772fce22"}
Jul 16 20:50:59.803717 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:59.803484    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:50:59.826039 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:50:59.825984    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j" podStartSLOduration=1.825967671 podStartE2EDuration="1.825967671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:50:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:50:59.82278602 +0000 UTC m=+3023.970881442" watchObservedRunningTime="2026-07-16 20:50:59.825967671 +0000 UTC m=+3023.974063093"
Jul 16 20:51:01.060624 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:01.060596    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lvjxn_cc035797-bbb8-4f53-bad4-95d700ac5ce7/dns/0.log"
Jul 16 20:51:01.087931 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:01.087902    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lvjxn_cc035797-bbb8-4f53-bad4-95d700ac5ce7/kube-rbac-proxy/0.log"
Jul 16 20:51:01.114951 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:01.114921    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hc42k_e306eeec-10d8-4f48-9456-eb6f14e7cb62/dns-node-resolver/0.log"
Jul 16 20:51:01.773310 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:01.773275    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5jv2r_906b4ac7-3b5b-4fdd-8862-ded58ddeb564/node-ca/0.log"
Jul 16 20:51:03.081373 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:03.081335    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7rz45_c2eea5a7-b748-4e9a-91a2-10a6b82d87b9/discovery/0.log"
Jul 16 20:51:03.298159 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:03.298119    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nsfxk_a192e8a0-baa2-4783-a097-bb862e2859a2/payload-processing/0.log"
Jul 16 20:51:04.070984 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:04.070956    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-97vtr_de75ba51-01b2-48d4-b7f8-c0c803fa0eba/serve-healthcheck-canary/0.log"
Jul 16 20:51:04.659148 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:04.659117    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9t7nr_9247a48e-28a7-4b4d-aa29-9d6d65e943d8/kube-rbac-proxy/0.log"
Jul 16 20:51:04.679517 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:04.679494    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9t7nr_9247a48e-28a7-4b4d-aa29-9d6d65e943d8/exporter/0.log"
Jul 16 20:51:04.704075 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:04.704055    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9t7nr_9247a48e-28a7-4b4d-aa29-9d6d65e943d8/extractor/0.log"
Jul 16 20:51:05.819308 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:05.819282    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lw5rh/perf-node-gather-daemonset-t8x5j"
Jul 16 20:51:07.690032 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:07.690003    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64c594d8d4-9vdc5_dd439004-1886-4e2e-bdac-ee5c110a0a07/manager/0.log"
Jul 16 20:51:07.714227 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:07.714203    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64c594d8d4-9vdc5_dd439004-1886-4e2e-bdac-ee5c110a0a07/copy-manifests/0.log"
Jul 16 20:51:08.604766 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:08.604730    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7rz45_c2eea5a7-b748-4e9a-91a2-10a6b82d87b9/discovery/0.log"
Jul 16 20:51:08.759522 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:08.759417    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nsfxk_a192e8a0-baa2-4783-a097-bb862e2859a2/payload-processing/0.log"
Jul 16 20:51:09.170367 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:09.170339    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b8db6765-dds9b_20e16a21-db23-4613-aba4-27c1f832a509/manager/0.log"
Jul 16 20:51:09.224446 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:09.224402    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-5289b_8d851ff9-3382-45dc-b3d0-9b0c1c28d88e/openshift-lws-operator/0.log"
Jul 16 20:51:10.231820 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:10.231794    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7rz45_c2eea5a7-b748-4e9a-91a2-10a6b82d87b9/discovery/0.log"
Jul 16 20:51:10.401300 ip-10-0-140-236 kubenswrapper[2579]: I0716 20:51:10.401268    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nsfxk_a192e8a0-baa2-4783-a097-bb862e2859a2/payload-processing/0.log"