Jul 24 22:50:13.696558 ip-10-0-132-153 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 22:50:13.696589 ip-10-0-132-153 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 22:50:13.696600 ip-10-0-132-153 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 22:50:13.696930 ip-10-0-132-153 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 22:50:24.914394 ip-10-0-132-153 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 22:50:24.914409 ip-10-0-132-153 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 99d4b6ea1a6c4b86b5c20581d9c99cc2 --
Jul 24 22:52:22.610651 ip-10-0-132-153 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 22:52:22.980471 ip-10-0-132-153 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 22:52:22.980471 ip-10-0-132-153 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 22:52:22.980471 ip-10-0-132-153 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 22:52:22.980471 ip-10-0-132-153 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 22:52:22.980471 ip-10-0-132-153 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 22:52:22.981534 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.981444    2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 22:52:22.986395 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986372    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:22.986395 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986391    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:22.986395 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986395    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:22.986395 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986398    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:22.986395 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986402    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986405    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986409    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986411    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986414    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986417    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986419    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986422    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986425    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986427    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986430    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986432    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986435    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986438    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986440    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986443    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986445    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986448    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986451    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986454    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:22.986598 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986457    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986460    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986462    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986465    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986468    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986470    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986473    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986476    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986478    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986481    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986483    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986486    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986488    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986491    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986495    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986498    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986501    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986503    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986506    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:22.987073 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986509    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986511    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986514    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986517    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986519    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986522    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986526    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986530    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986534    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986537    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986540    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986543    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986546    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986548    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986553    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986556    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986559    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986562    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986564    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986567    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:22.987597 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986570    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986572    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986575    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986577    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986580    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986584    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986587    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986591    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986595    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986598    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986601    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986604    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986607    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986610    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986613    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986617    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986620    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986623    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986625    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:22.988081 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986628    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986631    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986634    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986637    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986640    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986642    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986645    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986647    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986650    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986653    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986655    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986658    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986660    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986663    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986666    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986669    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.986671    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987396    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987404    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987408    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:22.988549 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987411    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987414    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987416    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987419    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987421    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987424    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987427    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987430    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987433    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987436    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987438    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987441    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987444    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987446    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987450    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987452    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987455    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987459    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987461    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987464    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:22.989025 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987466    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987471    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987474    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987478    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987480    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987483    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987488    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987492    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987494    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987497    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987499    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987502    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987506    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987508    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987511    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987513    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987516    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987518    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987521    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987523    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:22.989529 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987526    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987528    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987531    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987533    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987536    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987538    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987540    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987544    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987547    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987549    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987552    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987555    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987557    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987560    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987562    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987565    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987567    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987569    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987575    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:22.990051 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987578    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987581    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987584    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987586    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987589    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987593    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987595    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987598    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987601    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987604    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987606    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987609    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987612    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987614    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987617    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987619    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987622    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987624    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987626    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987629    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:22.990541 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987632    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987635    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987638    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987640    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987643    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987646    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987648    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987651    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987653    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987656    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987658    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987661    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987663    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987666    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987668    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987671    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.987674    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988059    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988064    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:22.991039 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988066    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988069    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988072    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988075    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988077    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988080    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988082    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988085    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988088    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988091    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988093    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988096    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988099    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988101    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988104    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988107    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988109    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988111    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988114    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988117    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:22.991527 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988120    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988122    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988125    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988128    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988130    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988133    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988135    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988138    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988140    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988145    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988148    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988151    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988153    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988156    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988159    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988161    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988164    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988166    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988169    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:22.992019 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988171    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988174    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988176    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988179    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988182    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988184    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988187    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988189    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988193    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988196    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988198    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988201    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988203    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988206    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988208    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988211    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988213    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988216    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988219    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:22.992489 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988223    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988226    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988230    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988235    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988240    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988243    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988259    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988262    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988265    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988267    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988270    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988273    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988276    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988279    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988281    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988284    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988286    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988291    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988293    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988296    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:22.992966 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988298    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988301    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988305    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988308    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988311    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988314    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988316    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988319    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988322    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988324    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988327    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988330    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988334    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988337    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988340    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988344    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988350    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988353    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.988356    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988434    2568 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 22:52:22.993451 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988449    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988459    2568 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988465    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988472    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988477    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988482    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988486    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988489    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988492    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988496    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988499    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988503    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988505    2568 flags.go:64] FLAG: --cgroup-root=""
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988508    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988511    2568 flags.go:64] FLAG: --client-ca-file=""
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988515    2568 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988518    2568 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988523    2568 flags.go:64] FLAG: --cluster-domain=""
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988526    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988529    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988532    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988535    2568 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988539    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 22:52:22.993930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988542    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988545    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988549    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988552    2568 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988557    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988561    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988564    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988567    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988571    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988574    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988577    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988580    2568 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988583    2568 flags.go:64] FLAG: --enable-server="true"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988586    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988590    2568 flags.go:64] FLAG: --event-burst="100"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988593    2568 flags.go:64] FLAG: --event-qps="50"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988596    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988599    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988601    2568 flags.go:64] FLAG: --eviction-hard=""
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988605    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988608    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988611    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988614    2568 flags.go:64] FLAG: --eviction-soft=""
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988617    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988620    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 22:52:22.994497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988623    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988626    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988629    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988632    2568 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988635    2568 flags.go:64] FLAG: --feature-gates=""
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988639    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988642    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988645    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988648    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988651    2568 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988654    2568 flags.go:64] FLAG: --help="false"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988661    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-132-153.ec2.internal"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988666    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988669    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988672    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988675    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988679    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988682    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988685    2568 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988688    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988691    2568 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988694    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988697    2568 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988700    2568 flags.go:64] FLAG: --kube-reserved=""
Jul 24 22:52:22.995154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988703    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988705    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988709    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988711    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988715    2568 flags.go:64] FLAG: --lock-file=""
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988717    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988720    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988723    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988729    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988733    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988735    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988738    2568 flags.go:64] FLAG: --logging-format="text"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988741    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988744    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988747    2568 flags.go:64] FLAG: --manifest-url=""
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988750    2568 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988754    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988757    2568 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988761    2568 flags.go:64] FLAG: --max-pods="110"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988765    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988770    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988773    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988776    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988779    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988782    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 22:52:22.995777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988785    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988792    2568 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988795    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988798    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988801    2568 flags.go:64] FLAG: --pod-cidr=""
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988804    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988810    2568 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988813    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988816    2568 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988819    2568 flags.go:64] FLAG: --port="10250"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988822    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988825    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d7149bf38805f8e1"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988828    2568 flags.go:64] FLAG: --qos-reserved=""
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988831    2568 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988834    2568 flags.go:64] FLAG: --register-node="true"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988837    2568 flags.go:64] FLAG: --register-with-taints=""
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988840    2568 flags.go:64] FLAG: --registry-burst="10"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988843    2568 flags.go:64] FLAG: --registry-qps="5"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988846    2568 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988849    2568 flags.go:64] FLAG: --reserved-memory=""
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988852    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988855    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988858    2568 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988861    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988864    2568 flags.go:64] FLAG: --runonce="false"
Jul 24 22:52:22.996397 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988866    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988871    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988873    2568 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988878    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988880    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988883    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988886    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988890    2568 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988893    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988895    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988898    2568 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988901    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988904    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988907    2568 flags.go:64] FLAG: --system-cgroups=""
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988910    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988915    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988918    2568 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988921    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988925    2568 flags.go:64] FLAG: --tls-min-version=""
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988928    2568 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988930    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988933    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988936    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988939    2568 flags.go:64] FLAG: --v="2"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988943    2568 flags.go:64] FLAG: --version="false"
Jul 24 22:52:22.997005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988947    2568 flags.go:64] FLAG: --vmodule=""
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988951    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.988954    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989067    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989070    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989074    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989077    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989079    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989083    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989086    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989089    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989092    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989095    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989098    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989100    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989103    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989105    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989108    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989111    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989114    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:22.997633 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989116    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989119    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989122    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989124    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989126    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989129    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989132    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989134    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989137    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989140    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989144    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989147    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989150    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989154    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989156    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989159    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989162    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989165    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989168    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:22.998125 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989170    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989174    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989177    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989180    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989184    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989188    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989191    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989194    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989196    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989199    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989202    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989204    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989208    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989211    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989213    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989216    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989218    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989221    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989223    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:22.998617 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989226    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989228    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989231    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989233    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989236    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989239    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989242    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989244    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989266    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989269    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989271    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989274    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989277    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989279    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989283    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989285    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989289    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989292    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989295    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989298    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:22.999115 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989300    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989303    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989305    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989308    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989311    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989313    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989316    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989318    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989321    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989324    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989326    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989329    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989331    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989334    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989336    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989339    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989342    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989344    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989347    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989349    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:22.999830 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989352    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989355    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989358    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.989362    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.990073    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.997264    2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.997282    2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.997846    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.997855    2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997929    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997934    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997938    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997941    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997944    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:23.000323 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997947    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997950    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997953    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997956    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997958    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997961    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997963    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997966    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997969    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997972    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997974    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997977    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997980    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997982    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997985    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997987    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997990    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997993    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997996    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.997998    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:23.000675 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998001    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998004    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998006    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998010    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998014    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998016    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998019    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998021    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998024    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998027    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998029    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998032    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998035    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998037    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998040    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998042    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998045    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998047    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998050    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:23.001146 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998052    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998055    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998057    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998060    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998062    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998065    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998067    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998070    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998073    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998076    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998078    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998081    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998083    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998086    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998089    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998092    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998096    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998098    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998102    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998104    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:23.001628 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998107    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998110    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998112    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998115    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998117    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998120    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998122    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998125    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998128    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998130    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998133    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998137    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998141    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998144    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998147    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998150    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998154    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998158    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998160    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:23.002105 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998163    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998166    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998169    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998171    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998174    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998176    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998179    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998182    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998184    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998188    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998192    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998195    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998199    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998202    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998205    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:23.002668 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998208    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.998213    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998339    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998345    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998348    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998351    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998353    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998356    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998359    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998361    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998364    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998366    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998369    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998371    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998374    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 22:52:23.003049 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998376    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998379    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998381    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998384    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998386    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998389    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998392    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998394    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998396    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998399    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998401    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998404    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998407    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998409    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998412    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998415    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998417    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998420    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998422    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998425    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 22:52:23.003440 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998428    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998430    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998433    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998435    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998438    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998441    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998444    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998447    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998450    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998452    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998455    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998457    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998460    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998462    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998465    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998467    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998470    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998473    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998475    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998478    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 22:52:23.003922 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998480    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998483    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998485    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998488    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998491    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998493    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998496    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998499    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998502    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998504    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998507    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998509    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998513    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998517    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998520    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998522    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998525    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998528    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998531    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 22:52:23.004431 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998534    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998537    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998540    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998542    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998545    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998548    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998551    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998553    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998556    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998558    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998561    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998563    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998567    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998570    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998573    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998576    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998579    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998582    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998584    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998587    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 22:52:23.004884 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998589    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998592    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998595    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998597    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998600    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998602    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:22.998605    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.998610    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:22.998726    2568 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.000958    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.002392    2568 server.go:1019] "Starting client certificate rotation"
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.002494    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.002541    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 22:52:23.005382 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.004480    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 22:52:23.005708 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.005626    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 22:52:23.017937 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.017916    2568 log.go:25] "Validated CRI v1 runtime API"
Jul 24 22:52:23.020313 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.020298    2568 log.go:25] "Validated CRI v1 image API"
Jul 24 22:52:23.021832 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.021815    2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 22:52:23.024139 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.024115    2568 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b0d2d5d9-3954-4e41-b42d-c3c14b643dd9:/dev/nvme0n1p3 c67fdf6d-5a9f-4df9-8dfa-a1d5e794fa65:/dev/nvme0n1p4]
Jul 24 22:52:23.024196 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.024135    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 22:52:23.029187 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.029073    2568 manager.go:216] Machine: {Timestamp:2026-07-24 22:52:23.027887384 +0000 UTC m=+0.304920930 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100077 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c9512b438f232cae18f26510e7efb SystemUUID:ec2c9512-b438-f232-cae1-8f26510e7efb BootID:99d4b6ea-1a6c-4b86-b5c2-0581d9c99cc2 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d3:01:7e:8a:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d3:01:7e:8a:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:9c:e2:a9:4d:4c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 22:52:23.029187 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.029187    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 22:52:23.029311 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.029300    2568 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 22:52:23.031837 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.031822    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 22:52:23.032274 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.032231    2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 22:52:23.032407 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.032277    2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-153.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 22:52:23.033403 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.033393    2568 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 22:52:23.033440 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.033407    2568 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 22:52:23.033440 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.033414    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 22:52:23.034031 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.034021    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 22:52:23.035093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.035083    2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 22:52:23.037008 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.036997    2568 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 22:52:23.037126 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.037116    2568 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 22:52:23.037163 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.037152    2568 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 22:52:23.037191 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.037168    2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 22:52:23.037191 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.037186    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 22:52:23.037260 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.037195    2568 kubelet.go:398] "Adding apiserver pod source"
Jul 24 22:52:23.037260 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.037204    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 22:52:23.037605 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.037593    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 22:52:23.037676 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.037612    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 22:52:23.039640 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.039624    2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 22:52:23.041795 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.041782    2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 22:52:23.041834 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.041812    2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 22:52:23.043405 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043393    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 22:52:23.043462 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043409    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 22:52:23.043462 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043415    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 22:52:23.043462 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043421    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 22:52:23.043462 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043429    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 22:52:23.043462 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043436    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 22:52:23.043462 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043448    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 22:52:23.043462 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043454    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 22:52:23.043462 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043460    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 22:52:23.043462 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043466    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 22:52:23.043696 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043476    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 22:52:23.043696 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.043483    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 22:52:23.044067 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.044056    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 22:52:23.044098 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.044069    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 22:52:23.046049 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.046002    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-153.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 22:52:23.046199 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.046176    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-153.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 22:52:23.046237 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.046193    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 22:52:23.047457 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.047444    2568 server.go:1268] "Started kubelet"
Jul 24 22:52:23.047609 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.047541    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 22:52:23.050241 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.050191    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 22:52:23.050394 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.050361    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 22:52:23.050668 ip-10-0-132-153 systemd[1]: Started Kubernetes Kubelet.
Jul 24 22:52:23.052334 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.052314    2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 22:52:23.053942 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.053926    2568 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 22:52:23.055304 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.054389    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-153.ec2.internal.18c55b99b5053eda  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-153.ec2.internal,UID:ip-10-0-132-153.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-153.ec2.internal,},FirstTimestamp:2026-07-24 22:52:23.047413466 +0000 UTC m=+0.324447001,LastTimestamp:2026-07-24 22:52:23.047413466 +0000 UTC m=+0.324447001,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-153.ec2.internal,}"
Jul 24 22:52:23.056833 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.056812    2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 22:52:23.060091 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.060077    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 22:52:23.060159 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.060081    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 22:52:23.060415 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.060329    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.060837 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.060802    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 22:52:23.060837 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.060819    2568 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 22:52:23.060837 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.060837    2568 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 22:52:23.061021 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.060925    2568 factory.go:55] Registering systemd factory
Jul 24 22:52:23.061021 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.060938    2568 factory.go:223] Registration of the systemd container factory successfully
Jul 24 22:52:23.061021 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.060975    2568 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 22:52:23.061021 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.060985    2568 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 22:52:23.061170 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.061151    2568 factory.go:153] Registering CRI-O factory
Jul 24 22:52:23.061170 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.061161    2568 factory.go:223] Registration of the crio container factory successfully
Jul 24 22:52:23.061242 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.061223    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 22:52:23.061242 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.061260    2568 factory.go:103] Registering Raw factory
Jul 24 22:52:23.061374 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.061278    2568 manager.go:1196] Started watching for new ooms in manager
Jul 24 22:52:23.061756 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.061700    2568 manager.go:349] Starting recovery of all containers
Jul 24 22:52:23.062351 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.062308    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xzq7p"
Jul 24 22:52:23.070190 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.070158    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xzq7p"
Jul 24 22:52:23.070362 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.070342    2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-153.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 22:52:23.070435 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.070372    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 22:52:23.072532 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.072438    2568 manager.go:354] Recovery completed
Jul 24 22:52:23.076272 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.076243    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:23.078156 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.078142    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:23.078231 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.078168    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:23.078231 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.078178    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:23.078673 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.078657    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 22:52:23.078673 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.078672    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 22:52:23.078801 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.078675    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:23.078801 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.078690    2568 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 22:52:23.079394 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.079378    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:23.079467 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.079409    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:23.079467 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.079420    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:23.080345 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.080333    2568 policy_none.go:49] "None policy: Start"
Jul 24 22:52:23.080394 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.080349    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 22:52:23.080394 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.080361    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 22:52:23.081439 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.081428    2568 policy_none.go:47] "Start"
Jul 24 22:52:23.123780 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.123583    2568 manager.go:337] "Starting Device Plugin manager"
Jul 24 22:52:23.123865 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.123805    2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 22:52:23.123865 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.123818    2568 server.go:87] "Starting device plugin registration server"
Jul 24 22:52:23.124082 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.124071    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 22:52:23.124113 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.124086    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 22:52:23.124441 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.124263    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 22:52:23.124441 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.124359    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 22:52:23.124441 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.124370    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 22:52:23.124861 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.124839    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 22:52:23.124913 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.124897    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.225203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.225154    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:23.226463 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.226444    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:23.226574 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.226477    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:23.226574 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.226491    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:23.226574 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.226519    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.232628 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.232549    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.232628 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.232574    2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-153.ec2.internal\": node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.248068 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.248032    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.249910 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.249880    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 22:52:23.251165 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.251143    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 22:52:23.251243 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.251195    2568 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 22:52:23.251243 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.251221    2568 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 22:52:23.251324 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.251263    2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 22:52:23.262608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.262590    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:23.352812 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.352758    2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal"]
Jul 24 22:52:23.352987 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.352866    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:23.352987 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.352771    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.355379 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.355361    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:23.355497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.355390    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:23.355497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.355401    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:23.356639 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.356625    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:23.356759 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.356743    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.356798 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.356786    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:23.357461 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.357446    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:23.357532 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.357475    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:23.357532 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.357486    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:23.357532 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.357452    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:23.357635 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.357547    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:23.357635 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.357557    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:23.358738 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.358723    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.358805 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.358748    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 22:52:23.359476 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.359454    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 22:52:23.359554 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.359491    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 22:52:23.359554 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.359509    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeHasSufficientPID"
Jul 24 22:52:23.362296 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.362277    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee83a6278cefd64eeff713aca508b6e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal\" (UID: \"ee83a6278cefd64eeff713aca508b6e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.362413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.362302    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee83a6278cefd64eeff713aca508b6e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal\" (UID: \"ee83a6278cefd64eeff713aca508b6e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.362413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.362321    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e2c72bef2cb1e0ad183f8af7f40f58d5-config\") pod \"kube-apiserver-proxy-ip-10-0-132-153.ec2.internal\" (UID: \"e2c72bef2cb1e0ad183f8af7f40f58d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.386194 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.386166    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-153.ec2.internal\" not found" node="ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.391000 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.390983    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-153.ec2.internal\" not found" node="ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.453300 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.453263    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.462826 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.462800    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee83a6278cefd64eeff713aca508b6e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal\" (UID: \"ee83a6278cefd64eeff713aca508b6e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.462928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.462838    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee83a6278cefd64eeff713aca508b6e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal\" (UID: \"ee83a6278cefd64eeff713aca508b6e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.462928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.462857    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e2c72bef2cb1e0ad183f8af7f40f58d5-config\") pod \"kube-apiserver-proxy-ip-10-0-132-153.ec2.internal\" (UID: \"e2c72bef2cb1e0ad183f8af7f40f58d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.462928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.462915    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee83a6278cefd64eeff713aca508b6e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal\" (UID: \"ee83a6278cefd64eeff713aca508b6e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.463038 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.462998    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee83a6278cefd64eeff713aca508b6e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal\" (UID: \"ee83a6278cefd64eeff713aca508b6e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.463081 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.463041    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e2c72bef2cb1e0ad183f8af7f40f58d5-config\") pod \"kube-apiserver-proxy-ip-10-0-132-153.ec2.internal\" (UID: \"e2c72bef2cb1e0ad183f8af7f40f58d5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.554668 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.554576    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.662371 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.662341    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.690725 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.690697    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.696564 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.696537    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:23.762543 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.762505    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.865511 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.865423    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.966464 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:23.966416    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:23.988996 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:23.988970    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:24.001715 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.001686    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 22:52:24.001840 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.001816    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 22:52:24.001880 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.001860    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 22:52:24.066990 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:24.066958    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:24.075833 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.075797    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 22:47:23 +0000 UTC" deadline="2028-01-01 17:12:55.943614859 +0000 UTC"
Jul 24 22:52:24.075833 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.075829    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12618h20m31.86778922s"
Jul 24 22:52:24.167781 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:24.167743    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-153.ec2.internal\" not found"
Jul 24 22:52:24.246105 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.246068    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:24.260813 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.260789    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:24.272411 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.272389    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 22:52:24.274125 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.274099    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal"
Jul 24 22:52:24.276547 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:24.276508    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c72bef2cb1e0ad183f8af7f40f58d5.slice/crio-9ff7ff3c03b7548ef9819e0107dde304cc125b97c5d6ce396afb1eba1cd16daf WatchSource:0}: Error finding container 9ff7ff3c03b7548ef9819e0107dde304cc125b97c5d6ce396afb1eba1cd16daf: Status 404 returned error can't find the container with id 9ff7ff3c03b7548ef9819e0107dde304cc125b97c5d6ce396afb1eba1cd16daf
Jul 24 22:52:24.276962 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:24.276940    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee83a6278cefd64eeff713aca508b6e3.slice/crio-c8a0e2e0a66884a541b65ee457a16e806eeb1a2e3ba155b0a551b9c8d371dc9f WatchSource:0}: Error finding container c8a0e2e0a66884a541b65ee457a16e806eeb1a2e3ba155b0a551b9c8d371dc9f: Status 404 returned error can't find the container with id c8a0e2e0a66884a541b65ee457a16e806eeb1a2e3ba155b0a551b9c8d371dc9f
Jul 24 22:52:24.282664 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.282649    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 22:52:24.282746 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.282677    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 22:52:24.536426 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.536393    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:24.824905 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:24.824805    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:25.038804 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.038766    2568 apiserver.go:52] "Watching apiserver"
Jul 24 22:52:25.048279 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.048249    2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 22:52:25.048903 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.048873    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-wvrhp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj","openshift-image-registry/node-ca-lxdcd","openshift-network-operator/iptables-alerter-q7smv","openshift-ovn-kubernetes/ovnkube-node-2c5sz","kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal","openshift-cluster-node-tuning-operator/tuned-2zzbg","openshift-dns/node-resolver-xllwh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal","openshift-multus/multus-additional-cni-plugins-vflcn","openshift-multus/multus-khwlx","openshift-multus/network-metrics-daemon-lvlzd","openshift-network-diagnostics/network-check-target-cmqnc","kube-system/global-pull-secret-syncer-cm8cs"]
Jul 24 22:52:25.050725 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.050695    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.051838 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.051817    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:25.051947 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.051897    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:25.053416 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.053392    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.054159 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.053905    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-q497r\""
Jul 24 22:52:25.054159 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.053943    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 22:52:25.054319 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.054160    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 22:52:25.057015 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.056214    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 22:52:25.057015 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.056559    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 22:52:25.057015 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.056655    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.057015 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.056665    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wqvwk\""
Jul 24 22:52:25.057015 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.056757    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 22:52:25.058469 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.057790    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.058805 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.058782    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 22:52:25.059163 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.059141    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 22:52:25.059393 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.059374    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 22:52:25.059690 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.059666    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qvd4w\""
Jul 24 22:52:25.059824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.059802    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:25.059956 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.059871    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:25.061281 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.060435    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 22:52:25.061281 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.060499    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 22:52:25.061281 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.060525    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t868z\""
Jul 24 22:52:25.061281 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.060669    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 22:52:25.061281 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.060737    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.061281 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.061175    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 22:52:25.061887 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.061841    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 22:52:25.061887 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.061866    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 22:52:25.061988 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.061923    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 22:52:25.062522 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.062500    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:25.063271 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.063249    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 22:52:25.063435 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.063416    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 22:52:25.063962 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.063942    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jpz2p\""
Jul 24 22:52:25.063962 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.063953    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.064881 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.064859    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8wdlh\""
Jul 24 22:52:25.064985 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.064964    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 22:52:25.065050 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.065024    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 22:52:25.065542 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.065518    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.067853 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.067320    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 22:52:25.067853 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.067319    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 22:52:25.069787 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.068008    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 22:52:25.069787 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.068206    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7mtcd\""
Jul 24 22:52:25.069787 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.068647    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 22:52:25.069787 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.069020    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-crqj6\""
Jul 24 22:52:25.069787 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.069029    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 22:52:25.070111 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.069833    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 22:52:25.070111 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.069979    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 22:52:25.070209 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.070120    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 22:52:25.070361 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.070328    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.072703 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072683    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.072809 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072720    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3273b38f-d584-4f4a-a7bc-ebf81b64af40-ovnkube-script-lib\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.072809 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072745    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xrkj\" (UniqueName: \"kubernetes.io/projected/3273b38f-d584-4f4a-a7bc-ebf81b64af40-kube-api-access-2xrkj\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.072809 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072773    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-sysconfig\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.072809 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072798    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-systemd\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.073025 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072823    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-run\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.073025 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072845    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-run-systemd\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.073025 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072894    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23064b8e-fd03-4d98-a8ee-aaa4c1e76521-iptables-alerter-script\") pod \"iptables-alerter-q7smv\" (UID: \"23064b8e-fd03-4d98-a8ee-aaa4c1e76521\") " pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.073025 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072927    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23064b8e-fd03-4d98-a8ee-aaa4c1e76521-host-slash\") pod \"iptables-alerter-q7smv\" (UID: \"23064b8e-fd03-4d98-a8ee-aaa4c1e76521\") " pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.073025 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072955    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3-konnectivity-ca\") pod \"konnectivity-agent-wvrhp\" (UID: \"a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3\") " pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:25.073025 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.072982    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-modprobe-d\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.073025 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073006    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-run-netns\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073031    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-run-ovn-kubernetes\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073055    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-cni-netd\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073080    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073128    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-sys\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073180    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6z4g\" (UniqueName: \"kubernetes.io/projected/64863c56-9777-45d6-8502-328507a7421e-kube-api-access-g6z4g\") pod \"node-ca-lxdcd\" (UID: \"64863c56-9777-45d6-8502-328507a7421e\") " pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073215    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-slash\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073222    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6nv4h\""
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073247    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-var-lib-openvswitch\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073268    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073287    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/105e164b-d539-4eb1-83e1-0db3b18886ca-kubelet-config\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073316    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-sysctl-d\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073341    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vbcl\" (UniqueName: \"kubernetes.io/projected/e10736f5-3859-45e8-bde0-30d6cf01defc-kube-api-access-9vbcl\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073364    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fac3b1fc-9561-4722-9701-abb84981e59b-hosts-file\") pod \"node-resolver-xllwh\" (UID: \"fac3b1fc-9561-4722-9701-abb84981e59b\") " pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073386    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64863c56-9777-45d6-8502-328507a7421e-host\") pod \"node-ca-lxdcd\" (UID: \"64863c56-9777-45d6-8502-328507a7421e\") " pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.073413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073422    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-kubelet\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073451    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-cni-bin\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073478    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3273b38f-d584-4f4a-a7bc-ebf81b64af40-ovnkube-config\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073536    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/105e164b-d539-4eb1-83e1-0db3b18886ca-dbus\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073564    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fac3b1fc-9561-4722-9701-abb84981e59b-tmp-dir\") pod \"node-resolver-xllwh\" (UID: \"fac3b1fc-9561-4722-9701-abb84981e59b\") " pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073587    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64863c56-9777-45d6-8502-328507a7421e-serviceca\") pod \"node-ca-lxdcd\" (UID: \"64863c56-9777-45d6-8502-328507a7421e\") " pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073611    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-systemd-units\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073654    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3273b38f-d584-4f4a-a7bc-ebf81b64af40-ovn-node-metrics-cert\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073690    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.073753    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073689    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3-agent-certs\") pod \"konnectivity-agent-wvrhp\" (UID: \"a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3\") " pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073802    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-kubernetes\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073834    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-lib-modules\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073858    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-tuned\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073882    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcpth\" (UniqueName: \"kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth\") pod \"network-check-target-cmqnc\" (UID: \"eca3657c-182c-47a7-84a7-d82ff5e23942\") " pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073904    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-run-ovn\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073927    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3273b38f-d584-4f4a-a7bc-ebf81b64af40-env-overrides\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.074128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073960    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-sysctl-conf\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.075023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.073983    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-host\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.075023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.074004    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e10736f5-3859-45e8-bde0-30d6cf01defc-tmp\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.075023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.074027    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfmjf\" (UniqueName: \"kubernetes.io/projected/fac3b1fc-9561-4722-9701-abb84981e59b-kube-api-access-kfmjf\") pod \"node-resolver-xllwh\" (UID: \"fac3b1fc-9561-4722-9701-abb84981e59b\") " pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.075023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.074053    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkk4q\" (UniqueName: \"kubernetes.io/projected/23064b8e-fd03-4d98-a8ee-aaa4c1e76521-kube-api-access-gkk4q\") pod \"iptables-alerter-q7smv\" (UID: \"23064b8e-fd03-4d98-a8ee-aaa4c1e76521\") " pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.075023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.074077    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-etc-openvswitch\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.075023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.074100    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-node-log\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.075023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.074124    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-var-lib-kubelet\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.075023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.074149    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-run-openvswitch\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.075023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.074172    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-log-socket\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.075604 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.075584    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 22:52:25.096438 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.096413    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lhn2x"
Jul 24 22:52:25.104871 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.104508    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lhn2x"
Jul 24 22:52:25.162052 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.162016    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 22:52:25.174493 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174459    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-etc-selinux\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.174672 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174503    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-cnibin\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.174672 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174533    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3-agent-certs\") pod \"konnectivity-agent-wvrhp\" (UID: \"a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3\") " pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:25.174672 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174599    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpth\" (UniqueName: \"kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth\") pod \"network-check-target-cmqnc\" (UID: \"eca3657c-182c-47a7-84a7-d82ff5e23942\") " pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:25.174672 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174641    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3273b38f-d584-4f4a-a7bc-ebf81b64af40-env-overrides\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.174672 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174669    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-conf-dir\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.174873 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174699    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-run-multus-certs\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.174873 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174724    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.174873 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174773    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfmjf\" (UniqueName: \"kubernetes.io/projected/fac3b1fc-9561-4722-9701-abb84981e59b-kube-api-access-kfmjf\") pod \"node-resolver-xllwh\" (UID: \"fac3b1fc-9561-4722-9701-abb84981e59b\") " pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.174873 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174801    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkk4q\" (UniqueName: \"kubernetes.io/projected/23064b8e-fd03-4d98-a8ee-aaa4c1e76521-kube-api-access-gkk4q\") pod \"iptables-alerter-q7smv\" (UID: \"23064b8e-fd03-4d98-a8ee-aaa4c1e76521\") " pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.174873 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174827    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-device-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.174873 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174854    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xrkj\" (UniqueName: \"kubernetes.io/projected/3273b38f-d584-4f4a-a7bc-ebf81b64af40-kube-api-access-2xrkj\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.175118 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174881    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-run-k8s-cni-cncf-io\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.175118 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.174933    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 22:52:25.175118 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175078    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-daemon-config\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.175118 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175112    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-run-systemd\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.175280 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175138    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-var-lib-cni-multus\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.175280 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175218    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nlt8\" (UniqueName: \"kubernetes.io/projected/57e80945-3acb-4559-bc9e-2c2c783f5a69-kube-api-access-2nlt8\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.175280 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175225    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3273b38f-d584-4f4a-a7bc-ebf81b64af40-env-overrides\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.175280 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175234    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-run-systemd\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.175280 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175241    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23064b8e-fd03-4d98-a8ee-aaa4c1e76521-iptables-alerter-script\") pod \"iptables-alerter-q7smv\" (UID: \"23064b8e-fd03-4d98-a8ee-aaa4c1e76521\") " pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.175558 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175388    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23064b8e-fd03-4d98-a8ee-aaa4c1e76521-host-slash\") pod \"iptables-alerter-q7smv\" (UID: \"23064b8e-fd03-4d98-a8ee-aaa4c1e76521\") " pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.175558 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175413    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-modprobe-d\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.175558 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175435    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-cni-netd\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.175558 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175467    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:25.175558 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175493    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-socket-dir-parent\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.175558 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175521    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-socket-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175571    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-sys\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175574    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-modprobe-d\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175593    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6z4g\" (UniqueName: \"kubernetes.io/projected/64863c56-9777-45d6-8502-328507a7421e-kube-api-access-g6z4g\") pod \"node-ca-lxdcd\" (UID: \"64863c56-9777-45d6-8502-328507a7421e\") " pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175611    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23064b8e-fd03-4d98-a8ee-aaa4c1e76521-host-slash\") pod \"iptables-alerter-q7smv\" (UID: \"23064b8e-fd03-4d98-a8ee-aaa4c1e76521\") " pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175618    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/105e164b-d539-4eb1-83e1-0db3b18886ca-kubelet-config\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175634    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-cni-netd\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175645    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-run-netns\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175704    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-sys\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.175787    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175788    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23064b8e-fd03-4d98-a8ee-aaa4c1e76521-iptables-alerter-script\") pod \"iptables-alerter-q7smv\" (UID: \"23064b8e-fd03-4d98-a8ee-aaa4c1e76521\") " pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175796    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175836    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/105e164b-d539-4eb1-83e1-0db3b18886ca-kubelet-config\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:25.175861 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175850    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8cce4f05-9038-4200-97d1-4bae3443d396-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.175885    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret podName:105e164b-d539-4eb1-83e1-0db3b18886ca nodeName:}" failed. No retries permitted until 2026-07-24 22:52:25.675864915 +0000 UTC m=+2.952898455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret") pod "global-pull-secret-syncer-cm8cs" (UID: "105e164b-d539-4eb1-83e1-0db3b18886ca") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175904    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vbcl\" (UniqueName: \"kubernetes.io/projected/e10736f5-3859-45e8-bde0-30d6cf01defc-kube-api-access-9vbcl\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.175929    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64863c56-9777-45d6-8502-328507a7421e-host\") pod \"node-ca-lxdcd\" (UID: \"64863c56-9777-45d6-8502-328507a7421e\") " pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176021    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-kubelet\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176056    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64863c56-9777-45d6-8502-328507a7421e-host\") pod \"node-ca-lxdcd\" (UID: \"64863c56-9777-45d6-8502-328507a7421e\") " pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176101    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-cni-bin\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176114    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-kubelet\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176140    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3273b38f-d584-4f4a-a7bc-ebf81b64af40-ovnkube-config\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176166    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-cni-bin\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176219    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-cni-dir\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176275    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-system-cni-dir\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176317    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-os-release\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176362    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fac3b1fc-9561-4722-9701-abb84981e59b-tmp-dir\") pod \"node-resolver-xllwh\" (UID: \"fac3b1fc-9561-4722-9701-abb84981e59b\") " pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176404    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64863c56-9777-45d6-8502-328507a7421e-serviceca\") pod \"node-ca-lxdcd\" (UID: \"64863c56-9777-45d6-8502-328507a7421e\") " pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176438    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8cce4f05-9038-4200-97d1-4bae3443d396-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.176524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176466    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-kubernetes\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176516    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-lib-modules\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176542    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-tuned\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176592    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-run-ovn\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176620    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-var-lib-cni-bin\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176670    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-var-lib-kubelet\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176689    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3273b38f-d584-4f4a-a7bc-ebf81b64af40-ovnkube-config\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176696    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-hostroot\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176770    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-sysctl-conf\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176799    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-host\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176826    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e10736f5-3859-45e8-bde0-30d6cf01defc-tmp\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176852    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-etc-openvswitch\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176878    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-node-log\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176918    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-log-socket\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176947    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nnqf\" (UniqueName: \"kubernetes.io/projected/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-kube-api-access-7nnqf\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.176974    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-sys-fs\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.177093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177042    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fac3b1fc-9561-4722-9701-abb84981e59b-tmp-dir\") pod \"node-resolver-xllwh\" (UID: \"fac3b1fc-9561-4722-9701-abb84981e59b\") " pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177144    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64863c56-9777-45d6-8502-328507a7421e-serviceca\") pod \"node-ca-lxdcd\" (UID: \"64863c56-9777-45d6-8502-328507a7421e\") " pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177323    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-kubernetes\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177435    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-lib-modules\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177461    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-host\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177473    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-etc-openvswitch\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177484    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-var-lib-kubelet\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177512    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-log-socket\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177513    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-run-openvswitch\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177528    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-node-log\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177547    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-run-ovn\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177548    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-sysctl-conf\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177571    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177592    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-var-lib-kubelet\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177606    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3273b38f-d584-4f4a-a7bc-ebf81b64af40-ovnkube-script-lib\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177638    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-run-openvswitch\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177645    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.177824 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177678    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-cni-binary-copy\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177798    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-sysconfig\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177885    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-systemd\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177896    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-sysconfig\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177930    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-run\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177959    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-systemd\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.177986    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-system-cni-dir\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178005    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-os-release\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178031    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3-konnectivity-ca\") pod \"konnectivity-agent-wvrhp\" (UID: \"a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3\") " pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178049    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-run-netns\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178067    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-run-ovn-kubernetes\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178104    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-run-ovn-kubernetes\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178118    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-registration-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178139    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-run\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178152    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178186    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-run-netns\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178186    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3273b38f-d584-4f4a-a7bc-ebf81b64af40-ovnkube-script-lib\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.178653 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178234    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-slash\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178268    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-var-lib-openvswitch\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178314    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-cnibin\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178336    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-var-lib-openvswitch\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178352    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8cce4f05-9038-4200-97d1-4bae3443d396-cni-binary-copy\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178378    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-host-slash\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178385    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-sysctl-d\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178458    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fac3b1fc-9561-4722-9701-abb84981e59b-hosts-file\") pod \"node-resolver-xllwh\" (UID: \"fac3b1fc-9561-4722-9701-abb84981e59b\") " pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178509    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/105e164b-d539-4eb1-83e1-0db3b18886ca-dbus\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178551    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnkhj\" (UniqueName: \"kubernetes.io/projected/3cc21e39-0f51-46a7-8104-61530151c730-kube-api-access-rnkhj\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178569    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3-konnectivity-ca\") pod \"konnectivity-agent-wvrhp\" (UID: \"a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3\") " pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178581    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkqzx\" (UniqueName: \"kubernetes.io/projected/8cce4f05-9038-4200-97d1-4bae3443d396-kube-api-access-xkqzx\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178512    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-sysctl-d\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178516    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fac3b1fc-9561-4722-9701-abb84981e59b-hosts-file\") pod \"node-resolver-xllwh\" (UID: \"fac3b1fc-9561-4722-9701-abb84981e59b\") " pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178613    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/105e164b-d539-4eb1-83e1-0db3b18886ca-dbus\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178614    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-systemd-units\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178710    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3273b38f-d584-4f4a-a7bc-ebf81b64af40-ovn-node-metrics-cert\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.179551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.178745    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-etc-kubernetes\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.180493 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.179433    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3-agent-certs\") pod \"konnectivity-agent-wvrhp\" (UID: \"a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3\") " pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:25.180493 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.180037    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e10736f5-3859-45e8-bde0-30d6cf01defc-etc-tuned\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.181290 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.180643    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3273b38f-d584-4f4a-a7bc-ebf81b64af40-systemd-units\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.181290 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.180660    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e10736f5-3859-45e8-bde0-30d6cf01defc-tmp\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.181674 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.181534    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3273b38f-d584-4f4a-a7bc-ebf81b64af40-ovn-node-metrics-cert\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.181793 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.181746    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:25.181793 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.181780    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:25.181793 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.181793    2568 projected.go:196] Error preparing data for projected volume kube-api-access-tcpth for pod openshift-network-diagnostics/network-check-target-cmqnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:25.181959 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.181868    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth podName:eca3657c-182c-47a7-84a7-d82ff5e23942 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:25.681841674 +0000 UTC m=+2.958875213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tcpth" (UniqueName: "kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth") pod "network-check-target-cmqnc" (UID: "eca3657c-182c-47a7-84a7-d82ff5e23942") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:25.184554 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.184527    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkk4q\" (UniqueName: \"kubernetes.io/projected/23064b8e-fd03-4d98-a8ee-aaa4c1e76521-kube-api-access-gkk4q\") pod \"iptables-alerter-q7smv\" (UID: \"23064b8e-fd03-4d98-a8ee-aaa4c1e76521\") " pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.187733 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.187545    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xrkj\" (UniqueName: \"kubernetes.io/projected/3273b38f-d584-4f4a-a7bc-ebf81b64af40-kube-api-access-2xrkj\") pod \"ovnkube-node-2c5sz\" (UID: \"3273b38f-d584-4f4a-a7bc-ebf81b64af40\") " pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.188209 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.188166    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6z4g\" (UniqueName: \"kubernetes.io/projected/64863c56-9777-45d6-8502-328507a7421e-kube-api-access-g6z4g\") pod \"node-ca-lxdcd\" (UID: \"64863c56-9777-45d6-8502-328507a7421e\") " pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.189456 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.189409    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfmjf\" (UniqueName: \"kubernetes.io/projected/fac3b1fc-9561-4722-9701-abb84981e59b-kube-api-access-kfmjf\") pod \"node-resolver-xllwh\" (UID: \"fac3b1fc-9561-4722-9701-abb84981e59b\") " pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.189456 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.189439    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vbcl\" (UniqueName: \"kubernetes.io/projected/e10736f5-3859-45e8-bde0-30d6cf01defc-kube-api-access-9vbcl\") pod \"tuned-2zzbg\" (UID: \"e10736f5-3859-45e8-bde0-30d6cf01defc\") " pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.256160 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.256100    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal" event={"ID":"ee83a6278cefd64eeff713aca508b6e3","Type":"ContainerStarted","Data":"c8a0e2e0a66884a541b65ee457a16e806eeb1a2e3ba155b0a551b9c8d371dc9f"}
Jul 24 22:52:25.257313 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.257264    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal" event={"ID":"e2c72bef2cb1e0ad183f8af7f40f58d5","Type":"ContainerStarted","Data":"9ff7ff3c03b7548ef9819e0107dde304cc125b97c5d6ce396afb1eba1cd16daf"}
Jul 24 22:52:25.279628 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.279600    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-cni-dir\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.279628 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.279630    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-system-cni-dir\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.279848 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.279654    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-os-release\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.279848 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.279760    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-cni-dir\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.279848 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.279784    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-system-cni-dir\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.279992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.279858    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-os-release\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.279992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.279895    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8cce4f05-9038-4200-97d1-4bae3443d396-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.279992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.279933    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-var-lib-cni-bin\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.279992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.279960    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-var-lib-kubelet\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.279992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.279985    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-hostroot\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280016    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nnqf\" (UniqueName: \"kubernetes.io/projected/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-kube-api-access-7nnqf\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280044    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-sys-fs\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.280240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280069    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-var-lib-kubelet\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280082    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-hostroot\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280104    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-sys-fs\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.280240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280124    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-var-lib-cni-bin\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280132    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-cni-binary-copy\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280166    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-system-cni-dir\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280191    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-os-release\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280221    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-registration-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280247    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280293    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-system-cni-dir\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280283    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-cnibin\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280343    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-cnibin\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280344    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-registration-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280351    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8cce4f05-9038-4200-97d1-4bae3443d396-cni-binary-copy\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280392    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnkhj\" (UniqueName: \"kubernetes.io/projected/3cc21e39-0f51-46a7-8104-61530151c730-kube-api-access-rnkhj\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280451    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkqzx\" (UniqueName: \"kubernetes.io/projected/8cce4f05-9038-4200-97d1-4bae3443d396-kube-api-access-xkqzx\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280492    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-etc-kubernetes\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280509    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280520    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8cce4f05-9038-4200-97d1-4bae3443d396-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280521    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-etc-selinux\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.280608 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280572    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-cnibin\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280616    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-conf-dir\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280626    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-etc-selinux\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280643    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-run-multus-certs\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280459    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-os-release\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280693    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-run-multus-certs\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280686    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-etc-kubernetes\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280695    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-conf-dir\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280725    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280748    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8cce4f05-9038-4200-97d1-4bae3443d396-cnibin\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280757    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-device-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280785    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-run-k8s-cni-cncf-io\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280799    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8cce4f05-9038-4200-97d1-4bae3443d396-cni-binary-copy\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280812    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-daemon-config\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280814    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280838    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-var-lib-cni-multus\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280864    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nlt8\" (UniqueName: \"kubernetes.io/projected/57e80945-3acb-4559-bc9e-2c2c783f5a69-kube-api-access-2nlt8\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.281203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280857    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-run-k8s-cni-cncf-io\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280842    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-device-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280909    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-var-lib-cni-multus\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280912    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-socket-dir-parent\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280959    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-socket-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280965    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-socket-dir-parent\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.280981    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-run-netns\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.281015    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.281030    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-host-run-netns\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.281037    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8cce4f05-9038-4200-97d1-4bae3443d396-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.281078    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57e80945-3acb-4559-bc9e-2c2c783f5a69-socket-dir\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.281124    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.281182    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs podName:3cc21e39-0f51-46a7-8104-61530151c730 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:25.781161461 +0000 UTC m=+3.058194984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs") pod "network-metrics-daemon-lvlzd" (UID: "3cc21e39-0f51-46a7-8104-61530151c730") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.281597    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8cce4f05-9038-4200-97d1-4bae3443d396-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.281612    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-cni-binary-copy\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.281915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.281655    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-multus-daemon-config\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.289379 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.289353    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nnqf\" (UniqueName: \"kubernetes.io/projected/fcfc80ed-fbac-4d6d-90ac-4d4f560782bc-kube-api-access-7nnqf\") pod \"multus-khwlx\" (UID: \"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc\") " pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.289379 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.289370    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnkhj\" (UniqueName: \"kubernetes.io/projected/3cc21e39-0f51-46a7-8104-61530151c730-kube-api-access-rnkhj\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:25.289598 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.289499    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkqzx\" (UniqueName: \"kubernetes.io/projected/8cce4f05-9038-4200-97d1-4bae3443d396-kube-api-access-xkqzx\") pod \"multus-additional-cni-plugins-vflcn\" (UID: \"8cce4f05-9038-4200-97d1-4bae3443d396\") " pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.290247 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.290222    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nlt8\" (UniqueName: \"kubernetes.io/projected/57e80945-3acb-4559-bc9e-2c2c783f5a69-kube-api-access-2nlt8\") pod \"aws-ebs-csi-driver-node-5gbtj\" (UID: \"57e80945-3acb-4559-bc9e-2c2c783f5a69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.366493 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.366405    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2zzbg"
Jul 24 22:52:25.376540 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.376507    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lxdcd"
Jul 24 22:52:25.387648 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.387620    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q7smv"
Jul 24 22:52:25.393541 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.393520    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:25.403303 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.403279    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xllwh"
Jul 24 22:52:25.411243 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.411224    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:25.419276 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.419242    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj"
Jul 24 22:52:25.429089 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.429066    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vflcn"
Jul 24 22:52:25.436035 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.436011    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-khwlx"
Jul 24 22:52:25.684157 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.684067    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:25.684345 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.684168    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpth\" (UniqueName: \"kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth\") pod \"network-check-target-cmqnc\" (UID: \"eca3657c-182c-47a7-84a7-d82ff5e23942\") " pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:25.684345 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.684243    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:25.684345 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.684315    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:25.684345 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.684339    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret podName:105e164b-d539-4eb1-83e1-0db3b18886ca nodeName:}" failed. No retries permitted until 2026-07-24 22:52:26.684313433 +0000 UTC m=+3.961346967 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret") pod "global-pull-secret-syncer-cm8cs" (UID: "105e164b-d539-4eb1-83e1-0db3b18886ca") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:25.684570 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.684337    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:25.684570 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.684391    2568 projected.go:196] Error preparing data for projected volume kube-api-access-tcpth for pod openshift-network-diagnostics/network-check-target-cmqnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:25.684570 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.684441    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth podName:eca3657c-182c-47a7-84a7-d82ff5e23942 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:26.684429003 +0000 UTC m=+3.961462528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcpth" (UniqueName: "kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth") pod "network-check-target-cmqnc" (UID: "eca3657c-182c-47a7-84a7-d82ff5e23942") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:25.785117 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:25.785075    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:25.785306 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.785210    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:25.785306 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:25.785302    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs podName:3cc21e39-0f51-46a7-8104-61530151c730 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:26.785263005 +0000 UTC m=+4.062296531 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs") pod "network-metrics-daemon-lvlzd" (UID: "3cc21e39-0f51-46a7-8104-61530151c730") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:26.017526 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:26.017490    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 22:52:26.106949 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:26.106910    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 22:47:25 +0000 UTC" deadline="2028-05-09 12:14:07.179471556 +0000 UTC"
Jul 24 22:52:26.106949 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:26.106946    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15709h21m41.072530479s"
Jul 24 22:52:26.251658 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:26.251622    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:26.251836 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:26.251622    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:26.251836 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:26.251750    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:26.251836 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:26.251824    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:26.303205 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:26.303160    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b51fe6_7dcd_4f8f_9a8f_4efe2d3a7cc3.slice/crio-8624829d892fdff785e6e925a8187f4f5ff1e5ae91f00a9b61a5981538fed4a3 WatchSource:0}: Error finding container 8624829d892fdff785e6e925a8187f4f5ff1e5ae91f00a9b61a5981538fed4a3: Status 404 returned error can't find the container with id 8624829d892fdff785e6e925a8187f4f5ff1e5ae91f00a9b61a5981538fed4a3
Jul 24 22:52:26.304474 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:26.304440    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10736f5_3859_45e8_bde0_30d6cf01defc.slice/crio-88bd3f15422724e0855f4edabc26123845c4db4f005bb18c6a0f8e34ce6d8f76 WatchSource:0}: Error finding container 88bd3f15422724e0855f4edabc26123845c4db4f005bb18c6a0f8e34ce6d8f76: Status 404 returned error can't find the container with id 88bd3f15422724e0855f4edabc26123845c4db4f005bb18c6a0f8e34ce6d8f76
Jul 24 22:52:26.305994 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:26.305935    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcfc80ed_fbac_4d6d_90ac_4d4f560782bc.slice/crio-786dc2f47703d37fadc781179c7a48ce5aed5b896811c467c073d2a5405a55b3 WatchSource:0}: Error finding container 786dc2f47703d37fadc781179c7a48ce5aed5b896811c467c073d2a5405a55b3: Status 404 returned error can't find the container with id 786dc2f47703d37fadc781179c7a48ce5aed5b896811c467c073d2a5405a55b3
Jul 24 22:52:26.310042 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:26.309986    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64863c56_9777_45d6_8502_328507a7421e.slice/crio-37f38be7bdbef420f665b94bdadd2963d744a531980f6235025b67a02402481c WatchSource:0}: Error finding container 37f38be7bdbef420f665b94bdadd2963d744a531980f6235025b67a02402481c: Status 404 returned error can't find the container with id 37f38be7bdbef420f665b94bdadd2963d744a531980f6235025b67a02402481c
Jul 24 22:52:26.311405 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:26.311153    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e80945_3acb_4559_bc9e_2c2c783f5a69.slice/crio-d5f2b624721b51f42bac051ff6e75ff37c4f59ae7911d1a8803bccbfd2959c28 WatchSource:0}: Error finding container d5f2b624721b51f42bac051ff6e75ff37c4f59ae7911d1a8803bccbfd2959c28: Status 404 returned error can't find the container with id d5f2b624721b51f42bac051ff6e75ff37c4f59ae7911d1a8803bccbfd2959c28
Jul 24 22:52:26.312262 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:26.312160    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23064b8e_fd03_4d98_a8ee_aaa4c1e76521.slice/crio-25b497e1b036ea0a981cf969c0693e39e581afa53cbbc3b9bf55a316c7cbc3d5 WatchSource:0}: Error finding container 25b497e1b036ea0a981cf969c0693e39e581afa53cbbc3b9bf55a316c7cbc3d5: Status 404 returned error can't find the container with id 25b497e1b036ea0a981cf969c0693e39e581afa53cbbc3b9bf55a316c7cbc3d5
Jul 24 22:52:26.316858 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:26.316605    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3273b38f_d584_4f4a_a7bc_ebf81b64af40.slice/crio-5ddea6a18af56716504a67e42595ce470e6b02c764b27c5ec88c387d16100110 WatchSource:0}: Error finding container 5ddea6a18af56716504a67e42595ce470e6b02c764b27c5ec88c387d16100110: Status 404 returned error can't find the container with id 5ddea6a18af56716504a67e42595ce470e6b02c764b27c5ec88c387d16100110
Jul 24 22:52:26.692016 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:26.691739    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpth\" (UniqueName: \"kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth\") pod \"network-check-target-cmqnc\" (UID: \"eca3657c-182c-47a7-84a7-d82ff5e23942\") " pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:26.692196 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:26.692053    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:26.692196 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:26.691947    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:26.692196 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:26.692123    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:26.692196 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:26.692138    2568 projected.go:196] Error preparing data for projected volume kube-api-access-tcpth for pod openshift-network-diagnostics/network-check-target-cmqnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:26.692196 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:26.692180    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:26.692196 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:26.692197    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth podName:eca3657c-182c-47a7-84a7-d82ff5e23942 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:28.692178995 +0000 UTC m=+5.969212520 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcpth" (UniqueName: "kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth") pod "network-check-target-cmqnc" (UID: "eca3657c-182c-47a7-84a7-d82ff5e23942") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:26.692479 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:26.692233    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret podName:105e164b-d539-4eb1-83e1-0db3b18886ca nodeName:}" failed. No retries permitted until 2026-07-24 22:52:28.692216944 +0000 UTC m=+5.969250495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret") pod "global-pull-secret-syncer-cm8cs" (UID: "105e164b-d539-4eb1-83e1-0db3b18886ca") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:26.792907 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:26.792853    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:26.793099 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:26.793025    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:26.793099 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:26.793094    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs podName:3cc21e39-0f51-46a7-8104-61530151c730 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:28.793073704 +0000 UTC m=+6.070107249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs") pod "network-metrics-daemon-lvlzd" (UID: "3cc21e39-0f51-46a7-8104-61530151c730") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:27.107993 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.107923    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 22:47:25 +0000 UTC" deadline="2027-12-24 02:40:43.976825907 +0000 UTC"
Jul 24 22:52:27.107993 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.107990    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12411h48m16.868840434s"
Jul 24 22:52:27.255455 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.254314    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:27.255455 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:27.254479    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:27.293099 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.292820    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2zzbg" event={"ID":"e10736f5-3859-45e8-bde0-30d6cf01defc","Type":"ContainerStarted","Data":"88bd3f15422724e0855f4edabc26123845c4db4f005bb18c6a0f8e34ce6d8f76"}
Jul 24 22:52:27.303163 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.303123    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vflcn" event={"ID":"8cce4f05-9038-4200-97d1-4bae3443d396","Type":"ContainerStarted","Data":"a0048cc39fcd5ad47d385723c34fe47b6a61f0676657095a1037b9b6934ae144"}
Jul 24 22:52:27.324059 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.323947    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj" event={"ID":"57e80945-3acb-4559-bc9e-2c2c783f5a69","Type":"ContainerStarted","Data":"d5f2b624721b51f42bac051ff6e75ff37c4f59ae7911d1a8803bccbfd2959c28"}
Jul 24 22:52:27.339506 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.339408    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lxdcd" event={"ID":"64863c56-9777-45d6-8502-328507a7421e","Type":"ContainerStarted","Data":"37f38be7bdbef420f665b94bdadd2963d744a531980f6235025b67a02402481c"}
Jul 24 22:52:27.345537 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.345450    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-khwlx" event={"ID":"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc","Type":"ContainerStarted","Data":"786dc2f47703d37fadc781179c7a48ce5aed5b896811c467c073d2a5405a55b3"}
Jul 24 22:52:27.364398 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.364289    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wvrhp" event={"ID":"a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3","Type":"ContainerStarted","Data":"8624829d892fdff785e6e925a8187f4f5ff1e5ae91f00a9b61a5981538fed4a3"}
Jul 24 22:52:27.382798 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.382632    2568 generic.go:356] "Generic (PLEG): container finished" podID="ee83a6278cefd64eeff713aca508b6e3" containerID="6176ea365b6d11d3481916c15f27f3440549a164063d772e2a2614c847f0984b" exitCode=0
Jul 24 22:52:27.382798 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.382735    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal" event={"ID":"ee83a6278cefd64eeff713aca508b6e3","Type":"ContainerDied","Data":"6176ea365b6d11d3481916c15f27f3440549a164063d772e2a2614c847f0984b"}
Jul 24 22:52:27.392677 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.392642    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal" event={"ID":"e2c72bef2cb1e0ad183f8af7f40f58d5","Type":"ContainerStarted","Data":"66a51dc35bbf5aacce16b082f3fc6b6a2c2b9243a58ce541a6c36d54e9d648d4"}
Jul 24 22:52:27.401093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.401059    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xllwh" event={"ID":"fac3b1fc-9561-4722-9701-abb84981e59b","Type":"ContainerStarted","Data":"8462a978b5c9192c76e08a500ba1e7406f1fd9e8ce0a75928e795f1bb558a685"}
Jul 24 22:52:27.418123 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.418085    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz" event={"ID":"3273b38f-d584-4f4a-a7bc-ebf81b64af40","Type":"ContainerStarted","Data":"5ddea6a18af56716504a67e42595ce470e6b02c764b27c5ec88c387d16100110"}
Jul 24 22:52:27.432770 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:27.432675    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q7smv" event={"ID":"23064b8e-fd03-4d98-a8ee-aaa4c1e76521","Type":"ContainerStarted","Data":"25b497e1b036ea0a981cf969c0693e39e581afa53cbbc3b9bf55a316c7cbc3d5"}
Jul 24 22:52:28.251968 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:28.251929    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:28.252470 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:28.252069    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:28.252534 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:28.252479    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:28.252598 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:28.252572    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:28.459882 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:28.459809    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal" event={"ID":"ee83a6278cefd64eeff713aca508b6e3","Type":"ContainerStarted","Data":"66f164071d5d7ef32fa9e16bc1ed79fa45bd49eb60011054a90f4355cccf828c"}
Jul 24 22:52:28.475725 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:28.474940    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-153.ec2.internal" podStartSLOduration=4.474917509 podStartE2EDuration="4.474917509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:27.417194495 +0000 UTC m=+4.694228040" watchObservedRunningTime="2026-07-24 22:52:28.474917509 +0000 UTC m=+5.751951057"
Jul 24 22:52:28.475725 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:28.475175    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-153.ec2.internal" podStartSLOduration=4.4751625520000005 podStartE2EDuration="4.475162552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:28.47515374 +0000 UTC m=+5.752187308" watchObservedRunningTime="2026-07-24 22:52:28.475162552 +0000 UTC m=+5.752196098"
Jul 24 22:52:28.709576 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:28.709527    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:28.709744 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:28.709627    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpth\" (UniqueName: \"kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth\") pod \"network-check-target-cmqnc\" (UID: \"eca3657c-182c-47a7-84a7-d82ff5e23942\") " pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:28.709825 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:28.709767    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:28.709825 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:28.709789    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:28.709825 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:28.709804    2568 projected.go:196] Error preparing data for projected volume kube-api-access-tcpth for pod openshift-network-diagnostics/network-check-target-cmqnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:28.709956 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:28.709862    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth podName:eca3657c-182c-47a7-84a7-d82ff5e23942 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:32.709843417 +0000 UTC m=+9.986877157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcpth" (UniqueName: "kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth") pod "network-check-target-cmqnc" (UID: "eca3657c-182c-47a7-84a7-d82ff5e23942") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:28.710455 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:28.710242    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:28.710455 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:28.710313    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret podName:105e164b-d539-4eb1-83e1-0db3b18886ca nodeName:}" failed. No retries permitted until 2026-07-24 22:52:32.710298969 +0000 UTC m=+9.987332496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret") pod "global-pull-secret-syncer-cm8cs" (UID: "105e164b-d539-4eb1-83e1-0db3b18886ca") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:28.810582 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:28.810540    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:28.810763 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:28.810698    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:28.810763 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:28.810762    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs podName:3cc21e39-0f51-46a7-8104-61530151c730 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:32.810743474 +0000 UTC m=+10.087777001 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs") pod "network-metrics-daemon-lvlzd" (UID: "3cc21e39-0f51-46a7-8104-61530151c730") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:29.263341 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:29.260381    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:29.263341 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:29.260557    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:30.252320 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:30.252287    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:30.252488 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:30.252443    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:30.252558 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:30.252504    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:30.252615 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:30.252604    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:31.251842 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:31.251811    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:31.252305 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:31.251930    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:32.252645 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:32.252422    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:32.252645 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:32.252446    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:32.252645 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:32.252602    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:32.253229 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:32.252688    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:32.747522 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:32.747488    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpth\" (UniqueName: \"kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth\") pod \"network-check-target-cmqnc\" (UID: \"eca3657c-182c-47a7-84a7-d82ff5e23942\") " pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:32.747707 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:32.747546    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:32.747707 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:32.747684    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:32.747837 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:32.747745    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret podName:105e164b-d539-4eb1-83e1-0db3b18886ca nodeName:}" failed. No retries permitted until 2026-07-24 22:52:40.74772814 +0000 UTC m=+18.024761683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret") pod "global-pull-secret-syncer-cm8cs" (UID: "105e164b-d539-4eb1-83e1-0db3b18886ca") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:32.748060 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:32.748023    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:32.748060 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:32.748057    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:32.748220 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:32.748071    2568 projected.go:196] Error preparing data for projected volume kube-api-access-tcpth for pod openshift-network-diagnostics/network-check-target-cmqnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:32.748220 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:32.748132    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth podName:eca3657c-182c-47a7-84a7-d82ff5e23942 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:40.748113654 +0000 UTC m=+18.025147186 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcpth" (UniqueName: "kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth") pod "network-check-target-cmqnc" (UID: "eca3657c-182c-47a7-84a7-d82ff5e23942") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:32.848064 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:32.848028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:32.848202 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:32.848176    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:32.848265 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:32.848242    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs podName:3cc21e39-0f51-46a7-8104-61530151c730 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:40.848225285 +0000 UTC m=+18.125258811 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs") pod "network-metrics-daemon-lvlzd" (UID: "3cc21e39-0f51-46a7-8104-61530151c730") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:33.252624 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:33.252584    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:33.252801 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:33.252708    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:34.252204 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:34.252169    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:34.252396 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:34.252169    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:34.252396 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:34.252319    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:34.252396 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:34.252374    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:35.252389 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:35.252355    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:35.252845 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:35.252507    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:36.251757 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:36.251723    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:36.251916 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:36.251734    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:36.251916 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:36.251823    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:36.251991 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:36.251918    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:37.255431 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:37.255399    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:37.255858 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:37.255529    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:38.252104 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:38.252062    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:38.252299 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:38.252191    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:38.252299 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:38.252265    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:38.252430 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:38.252371    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:39.252260 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:39.252213    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:39.252699 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:39.252379    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:40.252196 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:40.252169    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:40.252417 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:40.252170    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:40.252417 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:40.252288    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:40.252417 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:40.252340    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:40.808237 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:40.808198    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpth\" (UniqueName: \"kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth\") pod \"network-check-target-cmqnc\" (UID: \"eca3657c-182c-47a7-84a7-d82ff5e23942\") " pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:40.808237 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:40.808242    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:40.808595 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:40.808341    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:40.808595 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:40.808392    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret podName:105e164b-d539-4eb1-83e1-0db3b18886ca nodeName:}" failed. No retries permitted until 2026-07-24 22:52:56.808376744 +0000 UTC m=+34.085410266 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret") pod "global-pull-secret-syncer-cm8cs" (UID: "105e164b-d539-4eb1-83e1-0db3b18886ca") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:40.808595 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:40.808388    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:40.808595 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:40.808418    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:40.808595 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:40.808432    2568 projected.go:196] Error preparing data for projected volume kube-api-access-tcpth for pod openshift-network-diagnostics/network-check-target-cmqnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:40.808595 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:40.808490    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth podName:eca3657c-182c-47a7-84a7-d82ff5e23942 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:56.808474103 +0000 UTC m=+34.085507634 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcpth" (UniqueName: "kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth") pod "network-check-target-cmqnc" (UID: "eca3657c-182c-47a7-84a7-d82ff5e23942") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:40.909219 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:40.909185    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:40.909406 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:40.909364    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:40.909472 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:40.909432    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs podName:3cc21e39-0f51-46a7-8104-61530151c730 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:56.909414159 +0000 UTC m=+34.186447706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs") pod "network-metrics-daemon-lvlzd" (UID: "3cc21e39-0f51-46a7-8104-61530151c730") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:41.252042 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:41.252004    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:41.252212 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:41.252132    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:42.252107 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:42.252069    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:42.252615 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:42.252068    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:42.252615 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:42.252202    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:42.252615 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:42.252273    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:43.254383 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:43.254357    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:43.254791 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:43.254462    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:44.251946 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:44.251911    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:44.252146 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:44.251912    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:44.252146 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:44.252047    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:44.252146 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:44.252107    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:45.254022 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.253514    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:45.254022 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:45.253651    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:45.495771 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.495228    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2zzbg" event={"ID":"e10736f5-3859-45e8-bde0-30d6cf01defc","Type":"ContainerStarted","Data":"248b989374ef83a8331183d4045c38cc56fc830b9a41bced60686ce41b0b475f"}
Jul 24 22:52:45.496770 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.496748    2568 generic.go:356] "Generic (PLEG): container finished" podID="8cce4f05-9038-4200-97d1-4bae3443d396" containerID="825620f48082be7f27a2574bdd6a8e66bf776718b43bee4c631dd454d72a0e72" exitCode=0
Jul 24 22:52:45.496870 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.496816    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vflcn" event={"ID":"8cce4f05-9038-4200-97d1-4bae3443d396","Type":"ContainerDied","Data":"825620f48082be7f27a2574bdd6a8e66bf776718b43bee4c631dd454d72a0e72"}
Jul 24 22:52:45.498324 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.498299    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj" event={"ID":"57e80945-3acb-4559-bc9e-2c2c783f5a69","Type":"ContainerStarted","Data":"462127c1cbdecb403a00d3c32921165db9522f7442aabf66daa2a8438a343022"}
Jul 24 22:52:45.499785 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.499759    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lxdcd" event={"ID":"64863c56-9777-45d6-8502-328507a7421e","Type":"ContainerStarted","Data":"2a1068c06da18860efa348464560f2b113f83a46e59645759cb76b4261714424"}
Jul 24 22:52:45.501308 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.501285    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-khwlx" event={"ID":"fcfc80ed-fbac-4d6d-90ac-4d4f560782bc","Type":"ContainerStarted","Data":"d3e2811f6143c4b5cc432c109a186a16476923cecca0b27f1c55067b0d2b0caf"}
Jul 24 22:52:45.503238 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.503215    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wvrhp" event={"ID":"a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3","Type":"ContainerStarted","Data":"cf8f8779a34fe7e274f854a3875a37d9a180313bf6b8333e229d64eb4f003d08"}
Jul 24 22:52:45.504649 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.504623    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xllwh" event={"ID":"fac3b1fc-9561-4722-9701-abb84981e59b","Type":"ContainerStarted","Data":"3cbf1d41a159fcd8c861c702947673a5387497e00acddcce8a17007a53817b40"}
Jul 24 22:52:45.506789 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.506737    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz" event={"ID":"3273b38f-d584-4f4a-a7bc-ebf81b64af40","Type":"ContainerStarted","Data":"c160f7df51bb00cc499b0e84377a649937d481bdb09718869818f8d15bbbe1ec"}
Jul 24 22:52:45.506856 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.506810    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz" event={"ID":"3273b38f-d584-4f4a-a7bc-ebf81b64af40","Type":"ContainerStarted","Data":"3df4aff898b193a027c3ab4d4129523aecca4d62cbc6cc9571ce9140a75f040a"}
Jul 24 22:52:45.506916 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.506864    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz" event={"ID":"3273b38f-d584-4f4a-a7bc-ebf81b64af40","Type":"ContainerStarted","Data":"2f67884aa11d4f308d218419fae681ca00a6929b1ebbb8a898a9d5332aa0ccf0"}
Jul 24 22:52:45.531479 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.531426    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xllwh" podStartSLOduration=3.744581597 podStartE2EDuration="22.531411931s" totalImagesPullingTime="18.786830334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:45.531239328 +0000 UTC m=+22.808272871" watchObservedRunningTime="2026-07-24 22:52:45.531411931 +0000 UTC m=+22.808445475"
Jul 24 22:52:45.548773 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.548728    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-khwlx" podStartSLOduration=3.67255146 podStartE2EDuration="22.54871152s" totalImagesPullingTime="18.87616006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:45.548261461 +0000 UTC m=+22.825295026" watchObservedRunningTime="2026-07-24 22:52:45.54871152 +0000 UTC m=+22.825745063"
Jul 24 22:52:45.564823 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.564785    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2zzbg" podStartSLOduration=3.753263926 podStartE2EDuration="22.564770223s" totalImagesPullingTime="18.811506297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:45.564498467 +0000 UTC m=+22.841532011" watchObservedRunningTime="2026-07-24 22:52:45.564770223 +0000 UTC m=+22.841803766"
Jul 24 22:52:45.578100 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.578052    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lxdcd" podStartSLOduration=3.773909963 podStartE2EDuration="22.578036341s" totalImagesPullingTime="18.804126378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:45.577860839 +0000 UTC m=+22.854894396" watchObservedRunningTime="2026-07-24 22:52:45.578036341 +0000 UTC m=+22.855069885"
Jul 24 22:52:45.592065 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:45.592027    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wvrhp" podStartSLOduration=11.978882681 podStartE2EDuration="22.592011275s" totalImagesPullingTime="10.613128594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:45.59168151 +0000 UTC m=+22.868715074" watchObservedRunningTime="2026-07-24 22:52:45.592011275 +0000 UTC m=+22.869044818"
Jul 24 22:52:46.251433 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:46.251401    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:46.251433 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:46.251428    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:46.251735 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:46.251519    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:46.251735 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:46.251637    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:46.512557 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:46.512518    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz" event={"ID":"3273b38f-d584-4f4a-a7bc-ebf81b64af40","Type":"ContainerStarted","Data":"1e80c34caefd86ac584e8ed0a0aeede9098626b5d9046834ea37932ac5fedcb9"}
Jul 24 22:52:46.513489 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:46.512637    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz" event={"ID":"3273b38f-d584-4f4a-a7bc-ebf81b64af40","Type":"ContainerStarted","Data":"043fbe664c1cd62428e2f9aee64cc6ab3cd341c3f5bfbe1ac2bb89186bba47af"}
Jul 24 22:52:46.513489 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:46.512673    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz" event={"ID":"3273b38f-d584-4f4a-a7bc-ebf81b64af40","Type":"ContainerStarted","Data":"7f44741aae6b1489b1b44c1e7f04d2e66a12cf93960e2e47a126772e74f8d80d"}
Jul 24 22:52:46.514161 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:46.514133    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q7smv" event={"ID":"23064b8e-fd03-4d98-a8ee-aaa4c1e76521","Type":"ContainerStarted","Data":"4ef54ce47c38c074500ad8e0854364bfe4d2ea43e2df09c96b06c575b9e756d6"}
Jul 24 22:52:46.528905 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:46.528855    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q7smv" podStartSLOduration=4.730326557 podStartE2EDuration="23.528835851s" totalImagesPullingTime="18.798509294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:46.528482949 +0000 UTC m=+23.805516493" watchObservedRunningTime="2026-07-24 22:52:46.528835851 +0000 UTC m=+23.805869401"
Jul 24 22:52:46.581048 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:46.581018    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 22:52:47.137436 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:47.137330    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T22:52:46.581042234Z","UUID":"c83661ab-4894-42d1-811e-b17ecf46ed39","Handler":null,"Name":"","Endpoint":""}
Jul 24 22:52:47.138911 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:47.138882    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 22:52:47.139036 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:47.138960    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 22:52:47.251915 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:47.251881    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:47.252100 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:47.252032    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:47.351983 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:47.351939    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:47.352732 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:47.352711    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:47.518601 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:47.518567    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj" event={"ID":"57e80945-3acb-4559-bc9e-2c2c783f5a69","Type":"ContainerStarted","Data":"0db729e12b0b344b9ee9ce77e063abb2cccd7995e1da47bc767062ad41216640"}
Jul 24 22:52:47.519096 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:47.518865    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:47.519423 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:47.519406    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wvrhp"
Jul 24 22:52:48.252473 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:48.252441    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:48.252671 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:48.252482    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:48.252671 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:48.252564    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:48.252791 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:48.252671    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:48.523195 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:48.523070    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj" event={"ID":"57e80945-3acb-4559-bc9e-2c2c783f5a69","Type":"ContainerStarted","Data":"a63987db85465e15872f5da195ca6fddc1b78c7ac8ae61dede7f828169e3d91b"}
Jul 24 22:52:48.529504 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:48.527790    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz" event={"ID":"3273b38f-d584-4f4a-a7bc-ebf81b64af40","Type":"ContainerStarted","Data":"6baf971bc6bbed91bdd1db10a86963408821317b39fe7bcba12b9b9294c1cbc2"}
Jul 24 22:52:49.252279 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:49.252046    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:49.252455 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:49.252409    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:50.252435 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:50.252403    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:50.253334 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:50.252403    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:50.253334 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:50.252534    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:50.253334 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:50.252580    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:50.534189 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:50.534155    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz" event={"ID":"3273b38f-d584-4f4a-a7bc-ebf81b64af40","Type":"ContainerStarted","Data":"4b1fafd25e266881041023396c6ba43f04fa8a34bfc80b00248c7496cf2ce2d0"}
Jul 24 22:52:50.534463 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:50.534447    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:50.536026 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:50.536004    2568 generic.go:356] "Generic (PLEG): container finished" podID="8cce4f05-9038-4200-97d1-4bae3443d396" containerID="d0654d69f89c88b0e3533ca2c58bff4f512811bcb97f7afb8aa7264c997ef769" exitCode=0
Jul 24 22:52:50.536137 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:50.536043    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vflcn" event={"ID":"8cce4f05-9038-4200-97d1-4bae3443d396","Type":"ContainerDied","Data":"d0654d69f89c88b0e3533ca2c58bff4f512811bcb97f7afb8aa7264c997ef769"}
Jul 24 22:52:50.550233 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:50.550211    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:50.561532 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:50.561491    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5gbtj" podStartSLOduration=6.568329735 podStartE2EDuration="27.561476303s" totalImagesPullingTime="20.993146568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:48.544394028 +0000 UTC m=+25.821427573" watchObservedRunningTime="2026-07-24 22:52:50.561476303 +0000 UTC m=+27.838509846"
Jul 24 22:52:50.561708 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:50.561684    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz" podStartSLOduration=8.69232895 podStartE2EDuration="27.561678133s" totalImagesPullingTime="18.869349183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:50.560992069 +0000 UTC m=+27.838025614" watchObservedRunningTime="2026-07-24 22:52:50.561678133 +0000 UTC m=+27.838711677"
Jul 24 22:52:51.252132 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:51.251947    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:51.252330 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:51.252240    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:51.538617 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:51.538587    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:51.539063 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:51.538655    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:51.555053 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:51.555012    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:52:52.092681 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:52.092649    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cm8cs"]
Jul 24 22:52:52.092895 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:52.092797    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:52.092955 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:52.092919    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:52.095749 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:52.095726    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cmqnc"]
Jul 24 22:52:52.095889 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:52.095858    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:52.095986 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:52.095945    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:52.096596 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:52.096574    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lvlzd"]
Jul 24 22:52:52.096712 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:52.096666    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:52.096780 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:52.096765    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:52.542933 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:52.542893    2568 generic.go:356] "Generic (PLEG): container finished" podID="8cce4f05-9038-4200-97d1-4bae3443d396" containerID="90874dffff409705211e753e516081ebd1b1aebb88fcba789ae6fe70e650afaa" exitCode=0
Jul 24 22:52:52.543393 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:52.542980    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vflcn" event={"ID":"8cce4f05-9038-4200-97d1-4bae3443d396","Type":"ContainerDied","Data":"90874dffff409705211e753e516081ebd1b1aebb88fcba789ae6fe70e650afaa"}
Jul 24 22:52:53.252549 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:53.252515    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:53.252713 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:53.252608    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:54.252447 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:54.252244    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:54.252806 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:54.252269    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:54.252806 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:54.252517    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:54.252806 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:54.252598    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:54.550886 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:54.550855    2568 generic.go:356] "Generic (PLEG): container finished" podID="8cce4f05-9038-4200-97d1-4bae3443d396" containerID="d1c9477d7201087151491b52da6cf7e34ec9951de695bf2fc48cb8360bf2481e" exitCode=0
Jul 24 22:52:54.551056 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:54.550912    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vflcn" event={"ID":"8cce4f05-9038-4200-97d1-4bae3443d396","Type":"ContainerDied","Data":"d1c9477d7201087151491b52da6cf7e34ec9951de695bf2fc48cb8360bf2481e"}
Jul 24 22:52:55.252188 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:55.252154    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:55.252385 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:55.252301    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:56.252953 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:56.252374    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:56.252953 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:56.252409    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:56.252953 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:56.252538    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lvlzd" podUID="3cc21e39-0f51-46a7-8104-61530151c730"
Jul 24 22:52:56.252953 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:56.252636    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmqnc" podUID="eca3657c-182c-47a7-84a7-d82ff5e23942"
Jul 24 22:52:56.828022 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:56.827987    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpth\" (UniqueName: \"kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth\") pod \"network-check-target-cmqnc\" (UID: \"eca3657c-182c-47a7-84a7-d82ff5e23942\") " pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:56.828197 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:56.828029    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:56.828197 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:56.828161    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:56.828197 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:56.828185    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 22:52:56.828335 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:56.828205    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 22:52:56.828335 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:56.828216    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret podName:105e164b-d539-4eb1-83e1-0db3b18886ca nodeName:}" failed. No retries permitted until 2026-07-24 22:53:28.828200228 +0000 UTC m=+66.105233750 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret") pod "global-pull-secret-syncer-cm8cs" (UID: "105e164b-d539-4eb1-83e1-0db3b18886ca") : object "kube-system"/"original-pull-secret" not registered
Jul 24 22:52:56.828335 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:56.828217    2568 projected.go:196] Error preparing data for projected volume kube-api-access-tcpth for pod openshift-network-diagnostics/network-check-target-cmqnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:56.828335 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:56.828271    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth podName:eca3657c-182c-47a7-84a7-d82ff5e23942 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:28.828243941 +0000 UTC m=+66.105277497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tcpth" (UniqueName: "kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth") pod "network-check-target-cmqnc" (UID: "eca3657c-182c-47a7-84a7-d82ff5e23942") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 22:52:56.928614 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:56.928582    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:56.928799 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:56.928702    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:56.928799 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:56.928753    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs podName:3cc21e39-0f51-46a7-8104-61530151c730 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:28.928740148 +0000 UTC m=+66.205773670 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs") pod "network-metrics-daemon-lvlzd" (UID: "3cc21e39-0f51-46a7-8104-61530151c730") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 22:52:57.252123 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:57.252075    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:57.252330 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:57.252207    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm8cs" podUID="105e164b-d539-4eb1-83e1-0db3b18886ca"
Jul 24 22:52:58.031001 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.030971    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-153.ec2.internal" event="NodeReady"
Jul 24 22:52:58.031578 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.031150    2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 22:52:58.071314 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.071231    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cfwc9"]
Jul 24 22:52:58.098692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.098621    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ggkzp"]
Jul 24 22:52:58.098856 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.098766    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.101372 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.101348    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 22:52:58.101372 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.101359    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j4zlm\""
Jul 24 22:52:58.101561 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.101364    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 22:52:58.111747 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.111726    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cfwc9"]
Jul 24 22:52:58.111845 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.111761    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ggkzp"]
Jul 24 22:52:58.111845 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.111837    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ggkzp"
Jul 24 22:52:58.115388 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.115180    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 22:52:58.115388 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.115193    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 22:52:58.115388 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.115215    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 22:52:58.115388 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.115244    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z8ljk\""
Jul 24 22:52:58.239166 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.239135    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f157f026-514d-4b81-8497-6b39a07b9c58-tmp-dir\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.239367 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.239194    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f157f026-514d-4b81-8497-6b39a07b9c58-config-volume\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.239367 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.239339    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08c1379f-90b8-41f6-b19c-1bb382937934-cert\") pod \"ingress-canary-ggkzp\" (UID: \"08c1379f-90b8-41f6-b19c-1bb382937934\") " pod="openshift-ingress-canary/ingress-canary-ggkzp"
Jul 24 22:52:58.239491 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.239376    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtvqp\" (UniqueName: \"kubernetes.io/projected/f157f026-514d-4b81-8497-6b39a07b9c58-kube-api-access-rtvqp\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.239491 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.239405    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.239491 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.239439    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5mw4\" (UniqueName: \"kubernetes.io/projected/08c1379f-90b8-41f6-b19c-1bb382937934-kube-api-access-l5mw4\") pod \"ingress-canary-ggkzp\" (UID: \"08c1379f-90b8-41f6-b19c-1bb382937934\") " pod="openshift-ingress-canary/ingress-canary-ggkzp"
Jul 24 22:52:58.252280 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.252236    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:52:58.252429 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.252286    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:52:58.255071 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.255010    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 22:52:58.255071 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.255053    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 22:52:58.255289 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.255014    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 22:52:58.255289 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.255113    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2njwm\""
Jul 24 22:52:58.255289 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.255113    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9dffr\""
Jul 24 22:52:58.340186 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.340093    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.340186 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.340128    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5mw4\" (UniqueName: \"kubernetes.io/projected/08c1379f-90b8-41f6-b19c-1bb382937934-kube-api-access-l5mw4\") pod \"ingress-canary-ggkzp\" (UID: \"08c1379f-90b8-41f6-b19c-1bb382937934\") " pod="openshift-ingress-canary/ingress-canary-ggkzp"
Jul 24 22:52:58.340186 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.340160    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f157f026-514d-4b81-8497-6b39a07b9c58-tmp-dir\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.340485 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:58.340367    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:52:58.340485 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.340380    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f157f026-514d-4b81-8497-6b39a07b9c58-config-volume\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.340485 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:58.340445    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls podName:f157f026-514d-4b81-8497-6b39a07b9c58 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:58.840424583 +0000 UTC m=+36.117458129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls") pod "dns-default-cfwc9" (UID: "f157f026-514d-4b81-8497-6b39a07b9c58") : secret "dns-default-metrics-tls" not found
Jul 24 22:52:58.340640 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.340513    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08c1379f-90b8-41f6-b19c-1bb382937934-cert\") pod \"ingress-canary-ggkzp\" (UID: \"08c1379f-90b8-41f6-b19c-1bb382937934\") " pod="openshift-ingress-canary/ingress-canary-ggkzp"
Jul 24 22:52:58.340640 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.340537    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtvqp\" (UniqueName: \"kubernetes.io/projected/f157f026-514d-4b81-8497-6b39a07b9c58-kube-api-access-rtvqp\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.340772 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.340512    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f157f026-514d-4b81-8497-6b39a07b9c58-tmp-dir\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.340888 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.340869    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f157f026-514d-4b81-8497-6b39a07b9c58-config-volume\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.345321 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.345298    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08c1379f-90b8-41f6-b19c-1bb382937934-cert\") pod \"ingress-canary-ggkzp\" (UID: \"08c1379f-90b8-41f6-b19c-1bb382937934\") " pod="openshift-ingress-canary/ingress-canary-ggkzp"
Jul 24 22:52:58.349326 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.349303    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtvqp\" (UniqueName: \"kubernetes.io/projected/f157f026-514d-4b81-8497-6b39a07b9c58-kube-api-access-rtvqp\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.349477 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.349416    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5mw4\" (UniqueName: \"kubernetes.io/projected/08c1379f-90b8-41f6-b19c-1bb382937934-kube-api-access-l5mw4\") pod \"ingress-canary-ggkzp\" (UID: \"08c1379f-90b8-41f6-b19c-1bb382937934\") " pod="openshift-ingress-canary/ingress-canary-ggkzp"
Jul 24 22:52:58.421938 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.421899    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ggkzp"
Jul 24 22:52:58.758532 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.758321    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ggkzp"]
Jul 24 22:52:58.845941 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:58.845903    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:58.846095 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:58.846057    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:52:58.846149 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:58.846128    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls podName:f157f026-514d-4b81-8497-6b39a07b9c58 nodeName:}" failed. No retries permitted until 2026-07-24 22:52:59.846111356 +0000 UTC m=+37.123144881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls") pod "dns-default-cfwc9" (UID: "f157f026-514d-4b81-8497-6b39a07b9c58") : secret "dns-default-metrics-tls" not found
Jul 24 22:52:58.850362 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:52:58.850332    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c1379f_90b8_41f6_b19c_1bb382937934.slice/crio-c4670026db73975bd2f3601abd73abc9a8c3611f17bc229aa4afa7c7ba2870fd WatchSource:0}: Error finding container c4670026db73975bd2f3601abd73abc9a8c3611f17bc229aa4afa7c7ba2870fd: Status 404 returned error can't find the container with id c4670026db73975bd2f3601abd73abc9a8c3611f17bc229aa4afa7c7ba2870fd
Jul 24 22:52:59.252369 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:59.252334    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:52:59.255202 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:59.255176    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 22:52:59.567316 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:59.567220    2568 generic.go:356] "Generic (PLEG): container finished" podID="8cce4f05-9038-4200-97d1-4bae3443d396" containerID="791200243c0452c759d7570a466fbfa7db22ad0c9cc39eed78344e76ebc715cc" exitCode=0
Jul 24 22:52:59.567477 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:59.567339    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vflcn" event={"ID":"8cce4f05-9038-4200-97d1-4bae3443d396","Type":"ContainerDied","Data":"791200243c0452c759d7570a466fbfa7db22ad0c9cc39eed78344e76ebc715cc"}
Jul 24 22:52:59.568648 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:59.568616    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ggkzp" event={"ID":"08c1379f-90b8-41f6-b19c-1bb382937934","Type":"ContainerStarted","Data":"c4670026db73975bd2f3601abd73abc9a8c3611f17bc229aa4afa7c7ba2870fd"}
Jul 24 22:52:59.854124 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:52:59.854048    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:52:59.854287 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:59.854166    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:52:59.854287 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:52:59.854225    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls podName:f157f026-514d-4b81-8497-6b39a07b9c58 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:01.854209243 +0000 UTC m=+39.131242764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls") pod "dns-default-cfwc9" (UID: "f157f026-514d-4b81-8497-6b39a07b9c58") : secret "dns-default-metrics-tls" not found
Jul 24 22:53:00.574747 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:00.574706    2568 generic.go:356] "Generic (PLEG): container finished" podID="8cce4f05-9038-4200-97d1-4bae3443d396" containerID="0787f12cbc19fc2bd29b8a053a40669f031bf104eb985ef47f4df1f553f276d3" exitCode=0
Jul 24 22:53:00.575201 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:00.574766    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vflcn" event={"ID":"8cce4f05-9038-4200-97d1-4bae3443d396","Type":"ContainerDied","Data":"0787f12cbc19fc2bd29b8a053a40669f031bf104eb985ef47f4df1f553f276d3"}
Jul 24 22:53:01.477510 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.477324    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"]
Jul 24 22:53:01.479424 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.479408    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.480392 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.480368    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"]
Jul 24 22:53:01.483928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.482437    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:01.483928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.482453    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-dzg8d\""
Jul 24 22:53:01.483928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.483325    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 24 22:53:01.483928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.483381    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 24 22:53:01.483928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.483660    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 24 22:53:01.483928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.483676    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 24 22:53:01.485554 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.485537    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 24 22:53:01.487023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.487004    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 24 22:53:01.487136 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.487103    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qp6pq\""
Jul 24 22:53:01.489438 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.489415    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 24 22:53:01.490945 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.490921    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"]
Jul 24 22:53:01.499339 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.499320    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"]
Jul 24 22:53:01.568942 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.568912    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5-config\") pod \"service-ca-operator-5d6c75f84f-wwwqv\" (UID: \"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.569090 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.568985    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbzbh\" (UniqueName: \"kubernetes.io/projected/0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5-kube-api-access-gbzbh\") pod \"service-ca-operator-5d6c75f84f-wwwqv\" (UID: \"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.569090 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.569052    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5-serving-cert\") pod \"service-ca-operator-5d6c75f84f-wwwqv\" (UID: \"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.580395 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.580365    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vflcn" event={"ID":"8cce4f05-9038-4200-97d1-4bae3443d396","Type":"ContainerStarted","Data":"8e86dcee1760b467149870e950e48e24b704d13c5c81cb27a0c18321ea1885ee"}
Jul 24 22:53:01.581787 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.581764    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ggkzp" event={"ID":"08c1379f-90b8-41f6-b19c-1bb382937934","Type":"ContainerStarted","Data":"1f4d62d6ab7649d480503e150b0fa8425aa4d2d3bcf59a6a7113a077d375299b"}
Jul 24 22:53:01.583838 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.583812    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-dcd89596d-77ljd"]
Jul 24 22:53:01.585916 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.585902    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.588519 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.588505    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 22:53:01.588885 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.588869    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 22:53:01.588988 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.588971    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6tmxk\""
Jul 24 22:53:01.594725 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.594705    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 22:53:01.595280 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.595266    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 22:53:01.606460 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.606431    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dcd89596d-77ljd"]
Jul 24 22:53:01.614502 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.614456    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vflcn" podStartSLOduration=8.929425672 podStartE2EDuration="38.614440231s" totalImagesPullingTime="29.685014559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:01.613654371 +0000 UTC m=+38.890687915" watchObservedRunningTime="2026-07-24 22:53:01.614440231 +0000 UTC m=+38.891473768"
Jul 24 22:53:01.654849 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.654800    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ggkzp" podStartSLOduration=1.670754696 podStartE2EDuration="3.654781191s" totalImagesPullingTime="1.984026495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:01.65409562 +0000 UTC m=+38.931129164" watchObservedRunningTime="2026-07-24 22:53:01.654781191 +0000 UTC m=+38.931814739"
Jul 24 22:53:01.676269 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.669948    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbzbh\" (UniqueName: \"kubernetes.io/projected/0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5-kube-api-access-gbzbh\") pod \"service-ca-operator-5d6c75f84f-wwwqv\" (UID: \"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.676269 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.670004    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g24t4\" (UniqueName: \"kubernetes.io/projected/3a16c749-1890-4305-bf70-fbb828e8b523-kube-api-access-g24t4\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:01.676269 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.670047    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5-serving-cert\") pod \"service-ca-operator-5d6c75f84f-wwwqv\" (UID: \"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.676269 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.670091    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:01.676269 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.670126    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5-config\") pod \"service-ca-operator-5d6c75f84f-wwwqv\" (UID: \"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.676269 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.670853    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5-config\") pod \"service-ca-operator-5d6c75f84f-wwwqv\" (UID: \"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.676662 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.676592    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5-serving-cert\") pod \"service-ca-operator-5d6c75f84f-wwwqv\" (UID: \"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.683268 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.679290    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-m78lj"]
Jul 24 22:53:01.690274 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.685153    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.694692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.693753    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 24 22:53:01.694692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.694033    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 22:53:01.694692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.694225    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 22:53:01.694692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.694423    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nrr9s\""
Jul 24 22:53:01.694692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.694613    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 24 22:53:01.695014 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.694937    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbzbh\" (UniqueName: \"kubernetes.io/projected/0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5-kube-api-access-gbzbh\") pod \"service-ca-operator-5d6c75f84f-wwwqv\" (UID: \"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.696474 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.696447    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-m78lj"]
Jul 24 22:53:01.702591 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.701102    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 24 22:53:01.770765 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.770674    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.770765 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.770714    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-trusted-ca\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.770765 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.770736    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-bound-sa-token\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.770971 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.770815    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-certificates\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.770971 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.770902    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g24t4\" (UniqueName: \"kubernetes.io/projected/3a16c749-1890-4305-bf70-fbb828e8b523-kube-api-access-g24t4\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:01.770971 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.770962    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-installation-pull-secrets\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.771077 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.771008    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:01.771077 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.771057    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8s9s\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-kube-api-access-f8s9s\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.771161 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.771099    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-image-registry-private-configuration\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.771161 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:01.771108    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 22:53:01.771336 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.771169    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2235fbc2-c533-445c-8a54-8cb7234b9c39-ca-trust-extracted\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.771336 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:01.771182    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls podName:3a16c749-1890-4305-bf70-fbb828e8b523 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:02.271157282 +0000 UTC m=+39.548190816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls") pod "cluster-samples-operator-6748489656-2xxwn" (UID: "3a16c749-1890-4305-bf70-fbb828e8b523") : secret "samples-operator-tls" not found
Jul 24 22:53:01.780457 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.780430    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g24t4\" (UniqueName: \"kubernetes.io/projected/3a16c749-1890-4305-bf70-fbb828e8b523-kube-api-access-g24t4\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:01.793288 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.793267    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"
Jul 24 22:53:01.872373 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872292    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8s9s\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-kube-api-access-f8s9s\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.872373 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872346    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-image-registry-private-configuration\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.872373 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872370    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2235fbc2-c533-445c-8a54-8cb7234b9c39-ca-trust-extracted\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.872555 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872403    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7719c2-57e4-426d-8aee-742e2e535b19-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.872555 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872433    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.872555 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872456    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-trusted-ca\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.872555 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872482    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-bound-sa-token\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.872555 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872504    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e7719c2-57e4-426d-8aee-742e2e535b19-snapshots\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.872555 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872528    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e7719c2-57e4-426d-8aee-742e2e535b19-serving-cert\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.872866 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872559    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:53:01.872866 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872584    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-certificates\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.872866 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872631    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e7719c2-57e4-426d-8aee-742e2e535b19-tmp\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.872866 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872654    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgsdx\" (UniqueName: \"kubernetes.io/projected/9e7719c2-57e4-426d-8aee-742e2e535b19-kube-api-access-vgsdx\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.872866 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872691    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-installation-pull-secrets\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.872866 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.872718    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7719c2-57e4-426d-8aee-742e2e535b19-service-ca-bundle\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.872866 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:01.872816    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:53:01.873190 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:01.872895    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls podName:f157f026-514d-4b81-8497-6b39a07b9c58 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:05.872875211 +0000 UTC m=+43.149908759 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls") pod "dns-default-cfwc9" (UID: "f157f026-514d-4b81-8497-6b39a07b9c58") : secret "dns-default-metrics-tls" not found
Jul 24 22:53:01.873190 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:01.872904    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:53:01.873190 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:01.872923    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dcd89596d-77ljd: secret "image-registry-tls" not found
Jul 24 22:53:01.873190 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:01.872983    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls podName:2235fbc2-c533-445c-8a54-8cb7234b9c39 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:02.372965446 +0000 UTC m=+39.649998997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls") pod "image-registry-dcd89596d-77ljd" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39") : secret "image-registry-tls" not found
Jul 24 22:53:01.873417 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.873214    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2235fbc2-c533-445c-8a54-8cb7234b9c39-ca-trust-extracted\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.874676 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.874555    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-certificates\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.874816 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.874797    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-trusted-ca\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.876058 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.876038    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-image-registry-private-configuration\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.876611 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.876589    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-installation-pull-secrets\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.880876 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.880827    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8s9s\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-kube-api-access-f8s9s\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.881020 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.880990    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-bound-sa-token\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:01.921238 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.921209    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv"]
Jul 24 22:53:01.924965 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:01.924940    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a2a3cbd_ad24_4d6f_80b9_5ba68331bfa5.slice/crio-0394eeea287908700ca61c930e373aff704b4000d232a3a9719d888cfb62032b WatchSource:0}: Error finding container 0394eeea287908700ca61c930e373aff704b4000d232a3a9719d888cfb62032b: Status 404 returned error can't find the container with id 0394eeea287908700ca61c930e373aff704b4000d232a3a9719d888cfb62032b
Jul 24 22:53:01.973675 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.973486    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7719c2-57e4-426d-8aee-742e2e535b19-service-ca-bundle\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.973864 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.973745    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7719c2-57e4-426d-8aee-742e2e535b19-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.973864 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.973787    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e7719c2-57e4-426d-8aee-742e2e535b19-snapshots\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.973864 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.973809    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e7719c2-57e4-426d-8aee-742e2e535b19-serving-cert\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.974021 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.973869    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e7719c2-57e4-426d-8aee-742e2e535b19-tmp\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.974021 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.973892    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgsdx\" (UniqueName: \"kubernetes.io/projected/9e7719c2-57e4-426d-8aee-742e2e535b19-kube-api-access-vgsdx\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.974193 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.974173    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7719c2-57e4-426d-8aee-742e2e535b19-service-ca-bundle\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.974393 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.974368    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e7719c2-57e4-426d-8aee-742e2e535b19-tmp\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.974452 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.974434    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9e7719c2-57e4-426d-8aee-742e2e535b19-snapshots\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.974591 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.974575    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e7719c2-57e4-426d-8aee-742e2e535b19-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.976117 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.976100    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9e7719c2-57e4-426d-8aee-742e2e535b19-serving-cert\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:01.992850 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:01.992822    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgsdx\" (UniqueName: \"kubernetes.io/projected/9e7719c2-57e4-426d-8aee-742e2e535b19-kube-api-access-vgsdx\") pod \"insights-operator-5885fdb759-m78lj\" (UID: \"9e7719c2-57e4-426d-8aee-742e2e535b19\") " pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:02.005793 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:02.005764    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-m78lj"
Jul 24 22:53:02.126655 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:02.126619    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-m78lj"]
Jul 24 22:53:02.130472 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:02.130448    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e7719c2_57e4_426d_8aee_742e2e535b19.slice/crio-d5250f23e6668f73a431a84c04ac8f3a265000776095b711fbad1ccb7604da64 WatchSource:0}: Error finding container d5250f23e6668f73a431a84c04ac8f3a265000776095b711fbad1ccb7604da64: Status 404 returned error can't find the container with id d5250f23e6668f73a431a84c04ac8f3a265000776095b711fbad1ccb7604da64
Jul 24 22:53:02.276816 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:02.276783    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:02.276964 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:02.276937    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 22:53:02.277010 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:02.277000    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls podName:3a16c749-1890-4305-bf70-fbb828e8b523 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:03.276985593 +0000 UTC m=+40.554019114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls") pod "cluster-samples-operator-6748489656-2xxwn" (UID: "3a16c749-1890-4305-bf70-fbb828e8b523") : secret "samples-operator-tls" not found
Jul 24 22:53:02.377946 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:02.377857    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:02.378097 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:02.377973    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:53:02.378097 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:02.377984    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dcd89596d-77ljd: secret "image-registry-tls" not found
Jul 24 22:53:02.378097 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:02.378030    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls podName:2235fbc2-c533-445c-8a54-8cb7234b9c39 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:03.378017454 +0000 UTC m=+40.655050981 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls") pod "image-registry-dcd89596d-77ljd" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39") : secret "image-registry-tls" not found
Jul 24 22:53:02.585843 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:02.585804    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m78lj" event={"ID":"9e7719c2-57e4-426d-8aee-742e2e535b19","Type":"ContainerStarted","Data":"d5250f23e6668f73a431a84c04ac8f3a265000776095b711fbad1ccb7604da64"}
Jul 24 22:53:02.587085 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:02.587053    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv" event={"ID":"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5","Type":"ContainerStarted","Data":"0394eeea287908700ca61c930e373aff704b4000d232a3a9719d888cfb62032b"}
Jul 24 22:53:03.285501 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:03.285470    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:03.285680 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:03.285614    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 22:53:03.285720 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:03.285708    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls podName:3a16c749-1890-4305-bf70-fbb828e8b523 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:05.285689858 +0000 UTC m=+42.562723385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls") pod "cluster-samples-operator-6748489656-2xxwn" (UID: "3a16c749-1890-4305-bf70-fbb828e8b523") : secret "samples-operator-tls" not found
Jul 24 22:53:03.386346 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:03.386310    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:03.386503 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:03.386422    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:53:03.386503 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:03.386435    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dcd89596d-77ljd: secret "image-registry-tls" not found
Jul 24 22:53:03.386503 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:03.386488    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls podName:2235fbc2-c533-445c-8a54-8cb7234b9c39 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:05.386472571 +0000 UTC m=+42.663506094 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls") pod "image-registry-dcd89596d-77ljd" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39") : secret "image-registry-tls" not found
Jul 24 22:53:05.299636 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:05.299589    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:05.300092 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:05.299748    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 22:53:05.300092 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:05.299815    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls podName:3a16c749-1890-4305-bf70-fbb828e8b523 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:09.299799628 +0000 UTC m=+46.576833151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls") pod "cluster-samples-operator-6748489656-2xxwn" (UID: "3a16c749-1890-4305-bf70-fbb828e8b523") : secret "samples-operator-tls" not found
Jul 24 22:53:05.400547 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:05.400512    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:05.400737 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:05.400666    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:53:05.400737 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:05.400684    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dcd89596d-77ljd: secret "image-registry-tls" not found
Jul 24 22:53:05.400840 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:05.400745    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls podName:2235fbc2-c533-445c-8a54-8cb7234b9c39 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:09.400729596 +0000 UTC m=+46.677763126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls") pod "image-registry-dcd89596d-77ljd" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39") : secret "image-registry-tls" not found
Jul 24 22:53:05.596139 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:05.596037    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv" event={"ID":"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5","Type":"ContainerStarted","Data":"bdee97fe1070628597684e5db9b8a8de6ceaf3234f1c1b680600324def863632"}
Jul 24 22:53:05.597699 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:05.597660    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m78lj" event={"ID":"9e7719c2-57e4-426d-8aee-742e2e535b19","Type":"ContainerStarted","Data":"12d5305c9fe387d29b91090cb9807fcd9df282b41dc41ce6d07ea53041d8c516"}
Jul 24 22:53:05.611666 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:05.611610    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv" podStartSLOduration=1.622629037 podStartE2EDuration="4.611594961s" totalImagesPullingTime="2.988965924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:05.610698836 +0000 UTC m=+42.887732381" watchObservedRunningTime="2026-07-24 22:53:05.611594961 +0000 UTC m=+42.888628499"
Jul 24 22:53:05.628099 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:05.628042    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-m78lj" podStartSLOduration=1.840243912 podStartE2EDuration="4.628024095s" totalImagesPullingTime="2.787780183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:05.627288286 +0000 UTC m=+42.904321835" watchObservedRunningTime="2026-07-24 22:53:05.628024095 +0000 UTC m=+42.905057640"
Jul 24 22:53:05.904515 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:05.904416    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:53:05.904691 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:05.904557    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:53:05.904691 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:05.904627    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls podName:f157f026-514d-4b81-8497-6b39a07b9c58 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:13.904611562 +0000 UTC m=+51.181645087 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls") pod "dns-default-cfwc9" (UID: "f157f026-514d-4b81-8497-6b39a07b9c58") : secret "dns-default-metrics-tls" not found
Jul 24 22:53:06.246315 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.246281    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zqdf5"]
Jul 24 22:53:06.278589 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.278559    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zqdf5"]
Jul 24 22:53:06.278731 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.278667    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.281240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.281215    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 22:53:06.281240 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.281228    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 22:53:06.281436 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.281336    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mzp5b\""
Jul 24 22:53:06.409434 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.409388    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmdp5\" (UniqueName: \"kubernetes.io/projected/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-api-access-xmdp5\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.409434 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.409429    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-crio-socket\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.409882 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.409525    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.409882 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.409620    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.409882 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.409651    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-data-volume\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.510609 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.510508    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.510609 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.510584    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-data-volume\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.510826 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:06.510631    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 22:53:06.510826 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.510660    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmdp5\" (UniqueName: \"kubernetes.io/projected/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-api-access-xmdp5\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.510826 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.510688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-crio-socket\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.510826 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:06.510708    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm podName:17c261bd-15e5-4c8f-9cf6-8cab547c1c80 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:07.010686147 +0000 UTC m=+44.287719669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zqdf5" (UID: "17c261bd-15e5-4c8f-9cf6-8cab547c1c80") : configmap "kube-rbac-proxy" not found
Jul 24 22:53:06.510826 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.510778    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.510998 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.510866    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-crio-socket\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.510998 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:06.510897    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:06.510998 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.510939    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-data-volume\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:06.510998 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:06.510968    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls podName:17c261bd-15e5-4c8f-9cf6-8cab547c1c80 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:07.01095121 +0000 UTC m=+44.287984739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqdf5" (UID: "17c261bd-15e5-4c8f-9cf6-8cab547c1c80") : secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:06.520041 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:06.520015    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmdp5\" (UniqueName: \"kubernetes.io/projected/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-api-access-xmdp5\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:07.015163 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:07.015123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:07.015388 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:07.015177    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:07.015388 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:07.015277    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 22:53:07.015388 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:07.015326    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm podName:17c261bd-15e5-4c8f-9cf6-8cab547c1c80 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:08.015312791 +0000 UTC m=+45.292346312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zqdf5" (UID: "17c261bd-15e5-4c8f-9cf6-8cab547c1c80") : configmap "kube-rbac-proxy" not found
Jul 24 22:53:07.015388 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:07.015330    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:07.015557 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:07.015404    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls podName:17c261bd-15e5-4c8f-9cf6-8cab547c1c80 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:08.015386249 +0000 UTC m=+45.292419774 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqdf5" (UID: "17c261bd-15e5-4c8f-9cf6-8cab547c1c80") : secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:08.021887 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:08.021848    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:08.022438 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:08.021967    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 22:53:08.022438 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:08.021995    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:08.022438 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:08.022032    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm podName:17c261bd-15e5-4c8f-9cf6-8cab547c1c80 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:10.022016541 +0000 UTC m=+47.299050063 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zqdf5" (UID: "17c261bd-15e5-4c8f-9cf6-8cab547c1c80") : configmap "kube-rbac-proxy" not found
Jul 24 22:53:08.022438 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:08.022107    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:08.022438 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:08.022166    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls podName:17c261bd-15e5-4c8f-9cf6-8cab547c1c80 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:10.022148984 +0000 UTC m=+47.299182505 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqdf5" (UID: "17c261bd-15e5-4c8f-9cf6-8cab547c1c80") : secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:08.436778 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:08.436760    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xllwh_fac3b1fc-9561-4722-9701-abb84981e59b/dns-node-resolver/0.log"
Jul 24 22:53:09.331392 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:09.331355    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:09.331773 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:09.331507    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 22:53:09.331773 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:09.331570    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls podName:3a16c749-1890-4305-bf70-fbb828e8b523 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:17.331554201 +0000 UTC m=+54.608587722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls") pod "cluster-samples-operator-6748489656-2xxwn" (UID: "3a16c749-1890-4305-bf70-fbb828e8b523") : secret "samples-operator-tls" not found
Jul 24 22:53:09.432192 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:09.432157    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:09.432388 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:09.432282    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 22:53:09.432388 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:09.432295    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dcd89596d-77ljd: secret "image-registry-tls" not found
Jul 24 22:53:09.432388 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:09.432343    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls podName:2235fbc2-c533-445c-8a54-8cb7234b9c39 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:17.432328669 +0000 UTC m=+54.709362190 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls") pod "image-registry-dcd89596d-77ljd" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39") : secret "image-registry-tls" not found
Jul 24 22:53:09.433958 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:09.433938    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lxdcd_64863c56-9777-45d6-8502-328507a7421e/node-ca/0.log"
Jul 24 22:53:10.035027 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:10.035000    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ggkzp_08c1379f-90b8-41f6-b19c-1bb382937934/serve-healthcheck-canary/0.log"
Jul 24 22:53:10.038262 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:10.038223    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:10.038401 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:10.038288    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:10.038401 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:10.038366    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:10.038504 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:10.038438    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls podName:17c261bd-15e5-4c8f-9cf6-8cab547c1c80 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:14.038418394 +0000 UTC m=+51.315451929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqdf5" (UID: "17c261bd-15e5-4c8f-9cf6-8cab547c1c80") : secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:10.038736 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:10.038702    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:13.969199 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:13.969162    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:53:13.969597 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:13.969366    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 22:53:13.969597 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:13.969454    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls podName:f157f026-514d-4b81-8497-6b39a07b9c58 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:29.969433212 +0000 UTC m=+67.246466734 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls") pod "dns-default-cfwc9" (UID: "f157f026-514d-4b81-8497-6b39a07b9c58") : secret "dns-default-metrics-tls" not found
Jul 24 22:53:14.069581 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:14.069541    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:14.069744 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:14.069699    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:14.069783 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:14.069763    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls podName:17c261bd-15e5-4c8f-9cf6-8cab547c1c80 nodeName:}" failed. No retries permitted until 2026-07-24 22:53:22.069744808 +0000 UTC m=+59.346778337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqdf5" (UID: "17c261bd-15e5-4c8f-9cf6-8cab547c1c80") : secret "insights-runtime-extractor-tls" not found
Jul 24 22:53:17.397071 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:17.397031    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:17.399361 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:17.399332    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a16c749-1890-4305-bf70-fbb828e8b523-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-2xxwn\" (UID: \"3a16c749-1890-4305-bf70-fbb828e8b523\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:17.498022 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:17.497980    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:17.500306 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:17.500277    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls\") pod \"image-registry-dcd89596d-77ljd\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") " pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:17.698435 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:17.698407    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"
Jul 24 22:53:17.795669 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:17.795639    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:17.854445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:17.854418    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn"]
Jul 24 22:53:17.941702 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:17.941678    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dcd89596d-77ljd"]
Jul 24 22:53:17.944088 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:17.944062    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2235fbc2_c533_445c_8a54_8cb7234b9c39.slice/crio-546462938a456a422254ae6ae34755100194912453cdfbc05770b5cf229b6955 WatchSource:0}: Error finding container 546462938a456a422254ae6ae34755100194912453cdfbc05770b5cf229b6955: Status 404 returned error can't find the container with id 546462938a456a422254ae6ae34755100194912453cdfbc05770b5cf229b6955
Jul 24 22:53:18.629723 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:18.629689    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn" event={"ID":"3a16c749-1890-4305-bf70-fbb828e8b523","Type":"ContainerStarted","Data":"0e8a1cefff77d38f05b99677bf2f5094edf9281673ed1420f52095587efafa2a"}
Jul 24 22:53:18.631561 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:18.631531    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dcd89596d-77ljd" event={"ID":"2235fbc2-c533-445c-8a54-8cb7234b9c39","Type":"ContainerStarted","Data":"74e789771dcb41f7f7e8f43fc186aa41be6c54e8a85872167c8d9ccb1a355490"}
Jul 24 22:53:18.631699 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:18.631582    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dcd89596d-77ljd" event={"ID":"2235fbc2-c533-445c-8a54-8cb7234b9c39","Type":"ContainerStarted","Data":"546462938a456a422254ae6ae34755100194912453cdfbc05770b5cf229b6955"}
Jul 24 22:53:18.631765 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:18.631696    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:20.639647 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:20.639605    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn" event={"ID":"3a16c749-1890-4305-bf70-fbb828e8b523","Type":"ContainerStarted","Data":"76f8d84ee5c7c75abf113121d3781bf7557caa78cfbcdc9b04dd798930fb0d9e"}
Jul 24 22:53:20.640097 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:20.639662    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn" event={"ID":"3a16c749-1890-4305-bf70-fbb828e8b523","Type":"ContainerStarted","Data":"5f00a8bac167a8427e6fa8fdf82bc51038d663b8d4d23c0f4cb8a3fd43115bca"}
Jul 24 22:53:20.656799 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:20.656695    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-dcd89596d-77ljd" podStartSLOduration=19.656676193 podStartE2EDuration="19.656676193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:18.651265613 +0000 UTC m=+55.928299163" watchObservedRunningTime="2026-07-24 22:53:20.656676193 +0000 UTC m=+57.933709739"
Jul 24 22:53:20.657309 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:20.657264    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-2xxwn" podStartSLOduration=17.725734067 podStartE2EDuration="19.657236828s" totalImagesPullingTime="1.931502761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:20.655496322 +0000 UTC m=+57.932529868" watchObservedRunningTime="2026-07-24 22:53:20.657236828 +0000 UTC m=+57.934270375"
Jul 24 22:53:22.134982 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:22.134929    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:22.137418 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:22.137393    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17c261bd-15e5-4c8f-9cf6-8cab547c1c80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqdf5\" (UID: \"17c261bd-15e5-4c8f-9cf6-8cab547c1c80\") " pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:22.187551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:22.187518    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zqdf5"
Jul 24 22:53:22.307041 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:22.307018    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zqdf5"]
Jul 24 22:53:22.309631 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:22.309604    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c261bd_15e5_4c8f_9cf6_8cab547c1c80.slice/crio-99d68a927b9aa943881598a0df3d205368044cc71853de1a25ff8c7340f76892 WatchSource:0}: Error finding container 99d68a927b9aa943881598a0df3d205368044cc71853de1a25ff8c7340f76892: Status 404 returned error can't find the container with id 99d68a927b9aa943881598a0df3d205368044cc71853de1a25ff8c7340f76892
Jul 24 22:53:22.647399 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:22.647359    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqdf5" event={"ID":"17c261bd-15e5-4c8f-9cf6-8cab547c1c80","Type":"ContainerStarted","Data":"99d68a927b9aa943881598a0df3d205368044cc71853de1a25ff8c7340f76892"}
Jul 24 22:53:23.558290 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:23.558259    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2c5sz"
Jul 24 22:53:27.332674 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.332640    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"]
Jul 24 22:53:27.334853 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.334835    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.338482 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.338460    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 22:53:27.338627 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.338512    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 22:53:27.338892 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.338878    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 22:53:27.339791 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.339774    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 24 22:53:27.345182 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.345161    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dcd89596d-77ljd"]
Jul 24 22:53:27.363886 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.363821    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"]
Jul 24 22:53:27.375362 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.375335    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97becff6-40a8-4e7a-ab55-0d793f4209d2-tmp\") pod \"klusterlet-addon-workmgr-5c69d54cc6-wwt4x\" (UID: \"97becff6-40a8-4e7a-ab55-0d793f4209d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.375488 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.375453    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/97becff6-40a8-4e7a-ab55-0d793f4209d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c69d54cc6-wwt4x\" (UID: \"97becff6-40a8-4e7a-ab55-0d793f4209d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.375595 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.375570    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrpfj\" (UniqueName: \"kubernetes.io/projected/97becff6-40a8-4e7a-ab55-0d793f4209d2-kube-api-access-lrpfj\") pod \"klusterlet-addon-workmgr-5c69d54cc6-wwt4x\" (UID: \"97becff6-40a8-4e7a-ab55-0d793f4209d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.418271 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.418222    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-595bf88bc6-dj7ql"]
Jul 24 22:53:27.423069 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.423041    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.446208 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.446179    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-595bf88bc6-dj7ql"]
Jul 24 22:53:27.476468 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476408    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-registry-certificates\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.476468 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476468    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-bound-sa-token\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.476692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476494    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw8pl\" (UniqueName: \"kubernetes.io/projected/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-kube-api-access-xw8pl\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.476692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476519    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-registry-tls\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.476692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476550    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrpfj\" (UniqueName: \"kubernetes.io/projected/97becff6-40a8-4e7a-ab55-0d793f4209d2-kube-api-access-lrpfj\") pod \"klusterlet-addon-workmgr-5c69d54cc6-wwt4x\" (UID: \"97becff6-40a8-4e7a-ab55-0d793f4209d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.476692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476605    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97becff6-40a8-4e7a-ab55-0d793f4209d2-tmp\") pod \"klusterlet-addon-workmgr-5c69d54cc6-wwt4x\" (UID: \"97becff6-40a8-4e7a-ab55-0d793f4209d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.476844 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476761    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-image-registry-private-configuration\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.476844 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476814    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-installation-pull-secrets\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.476912 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476846    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/97becff6-40a8-4e7a-ab55-0d793f4209d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c69d54cc6-wwt4x\" (UID: \"97becff6-40a8-4e7a-ab55-0d793f4209d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.476912 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476866    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-ca-trust-extracted\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.476912 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.476880    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-trusted-ca\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.477126 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.477103    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97becff6-40a8-4e7a-ab55-0d793f4209d2-tmp\") pod \"klusterlet-addon-workmgr-5c69d54cc6-wwt4x\" (UID: \"97becff6-40a8-4e7a-ab55-0d793f4209d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.479783 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.479754    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/97becff6-40a8-4e7a-ab55-0d793f4209d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c69d54cc6-wwt4x\" (UID: \"97becff6-40a8-4e7a-ab55-0d793f4209d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.497511 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.497481    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrpfj\" (UniqueName: \"kubernetes.io/projected/97becff6-40a8-4e7a-ab55-0d793f4209d2-kube-api-access-lrpfj\") pod \"klusterlet-addon-workmgr-5c69d54cc6-wwt4x\" (UID: \"97becff6-40a8-4e7a-ab55-0d793f4209d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.578196 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.578160    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-bound-sa-token\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.578196 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.578200    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8pl\" (UniqueName: \"kubernetes.io/projected/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-kube-api-access-xw8pl\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.578453 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.578219    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-registry-tls\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.578453 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.578284    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-image-registry-private-configuration\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.578453 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.578333    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-installation-pull-secrets\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.578453 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.578395    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-ca-trust-extracted\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.578453 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.578419    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-trusted-ca\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.578724 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.578699    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-registry-certificates\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.578953 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.578923    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-ca-trust-extracted\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.579482 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.579463    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-registry-certificates\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.579599 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.579529    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-trusted-ca\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.581128 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.581095    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-image-registry-private-configuration\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.581273 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.581184    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-installation-pull-secrets\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.581273 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.581208    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-registry-tls\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.589886 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.589833    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-bound-sa-token\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.591201 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.591163    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw8pl\" (UniqueName: \"kubernetes.io/projected/5a2b5a24-16bd-4d43-b0e0-6bb5c117359e-kube-api-access-xw8pl\") pod \"image-registry-595bf88bc6-dj7ql\" (UID: \"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e\") " pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.645584 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.645508    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:27.668801 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.668764    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqdf5" event={"ID":"17c261bd-15e5-4c8f-9cf6-8cab547c1c80","Type":"ContainerStarted","Data":"58e0dcc34be5760de0ef9a58a3d57df615c52a0e7dc3e95e44e65d765ec02217"}
Jul 24 22:53:27.732213 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.732176    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:27.815280 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.813781    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"]
Jul 24 22:53:27.819850 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:27.819748    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97becff6_40a8_4e7a_ab55_0d793f4209d2.slice/crio-89a719e72159a1e7c6aed8cb158d787024a5347416c0ad60c0b542e45aaca21a WatchSource:0}: Error finding container 89a719e72159a1e7c6aed8cb158d787024a5347416c0ad60c0b542e45aaca21a: Status 404 returned error can't find the container with id 89a719e72159a1e7c6aed8cb158d787024a5347416c0ad60c0b542e45aaca21a
Jul 24 22:53:27.897470 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:27.897390    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-595bf88bc6-dj7ql"]
Jul 24 22:53:27.901579 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:27.901522    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2b5a24_16bd_4d43_b0e0_6bb5c117359e.slice/crio-86997c0def265e0c568f7fdf49dc5a59ea40f9aa67bbf8af3fca27a0c2a5827d WatchSource:0}: Error finding container 86997c0def265e0c568f7fdf49dc5a59ea40f9aa67bbf8af3fca27a0c2a5827d: Status 404 returned error can't find the container with id 86997c0def265e0c568f7fdf49dc5a59ea40f9aa67bbf8af3fca27a0c2a5827d
Jul 24 22:53:28.674150 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.674077    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x" event={"ID":"97becff6-40a8-4e7a-ab55-0d793f4209d2","Type":"ContainerStarted","Data":"89a719e72159a1e7c6aed8cb158d787024a5347416c0ad60c0b542e45aaca21a"}
Jul 24 22:53:28.676057 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.676022    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql" event={"ID":"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e","Type":"ContainerStarted","Data":"df18005f2a58c96e0f940ec72450131cf1b8f322ec36b325e45fa10157291242"}
Jul 24 22:53:28.676203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.676079    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql" event={"ID":"5a2b5a24-16bd-4d43-b0e0-6bb5c117359e","Type":"ContainerStarted","Data":"86997c0def265e0c568f7fdf49dc5a59ea40f9aa67bbf8af3fca27a0c2a5827d"}
Jul 24 22:53:28.676282 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.676234    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:28.695530 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.695483    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql" podStartSLOduration=1.695468155 podStartE2EDuration="1.695468155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:28.694270027 +0000 UTC m=+65.971303584" watchObservedRunningTime="2026-07-24 22:53:28.695468155 +0000 UTC m=+65.972501682"
Jul 24 22:53:28.891135 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.891090    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:53:28.891348 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.891171    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpth\" (UniqueName: \"kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth\") pod \"network-check-target-cmqnc\" (UID: \"eca3657c-182c-47a7-84a7-d82ff5e23942\") " pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:53:28.893871 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.893844    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 22:53:28.893996 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.893924    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 22:53:28.904180 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.904154    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 22:53:28.905033 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.904961    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/105e164b-d539-4eb1-83e1-0db3b18886ca-original-pull-secret\") pod \"global-pull-secret-syncer-cm8cs\" (UID: \"105e164b-d539-4eb1-83e1-0db3b18886ca\") " pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:53:28.915093 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.915061    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcpth\" (UniqueName: \"kubernetes.io/projected/eca3657c-182c-47a7-84a7-d82ff5e23942-kube-api-access-tcpth\") pod \"network-check-target-cmqnc\" (UID: \"eca3657c-182c-47a7-84a7-d82ff5e23942\") " pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:53:28.963088 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.963053    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm8cs"
Jul 24 22:53:28.991951 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.991918    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:53:28.994355 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:28.994326    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 22:53:29.004774 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:29.004748    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cc21e39-0f51-46a7-8104-61530151c730-metrics-certs\") pod \"network-metrics-daemon-lvlzd\" (UID: \"3cc21e39-0f51-46a7-8104-61530151c730\") " pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:53:29.167059 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:29.166840    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9dffr\""
Jul 24 22:53:29.172157 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:29.172099    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2njwm\""
Jul 24 22:53:29.175092 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:29.175052    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:53:29.180340 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:29.180213    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lvlzd"
Jul 24 22:53:29.999202 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:29.999167    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:53:30.001862 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:30.001827    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f157f026-514d-4b81-8497-6b39a07b9c58-metrics-tls\") pod \"dns-default-cfwc9\" (UID: \"f157f026-514d-4b81-8497-6b39a07b9c58\") " pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:53:30.212747 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:30.212716    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j4zlm\""
Jul 24 22:53:30.220900 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:30.220872    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:53:31.541520 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:31.541437    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cfwc9"]
Jul 24 22:53:31.687907 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:31.687873    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqdf5" event={"ID":"17c261bd-15e5-4c8f-9cf6-8cab547c1c80","Type":"ContainerStarted","Data":"08bd7b196fdd9ee6b3e5d73d559be1edab90b1009e337edf265bcc85c03aded9"}
Jul 24 22:53:31.688960 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:31.688933    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cfwc9" event={"ID":"f157f026-514d-4b81-8497-6b39a07b9c58","Type":"ContainerStarted","Data":"b53b4e5f0064fd3e97a33a3b6e790d8a1cfe27da8945e844b8605b64fc4688b8"}
Jul 24 22:53:31.762384 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:31.762345    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cm8cs"]
Jul 24 22:53:31.765941 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:31.765902    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod105e164b_d539_4eb1_83e1_0db3b18886ca.slice/crio-21122e506ce81aa2f5ef32724d8d5793ad607209d4050566bebf3921d9fa28cd WatchSource:0}: Error finding container 21122e506ce81aa2f5ef32724d8d5793ad607209d4050566bebf3921d9fa28cd: Status 404 returned error can't find the container with id 21122e506ce81aa2f5ef32724d8d5793ad607209d4050566bebf3921d9fa28cd
Jul 24 22:53:31.770377 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:31.769435    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cmqnc"]
Jul 24 22:53:31.770377 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:31.769533    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lvlzd"]
Jul 24 22:53:31.774903 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:31.774879    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc21e39_0f51_46a7_8104_61530151c730.slice/crio-caad4df7534d50bdb3537342248d629a7aadf5917c156e3a54b647ebefbdf78e WatchSource:0}: Error finding container caad4df7534d50bdb3537342248d629a7aadf5917c156e3a54b647ebefbdf78e: Status 404 returned error can't find the container with id caad4df7534d50bdb3537342248d629a7aadf5917c156e3a54b647ebefbdf78e
Jul 24 22:53:32.698643 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:32.698545    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lvlzd" event={"ID":"3cc21e39-0f51-46a7-8104-61530151c730","Type":"ContainerStarted","Data":"caad4df7534d50bdb3537342248d629a7aadf5917c156e3a54b647ebefbdf78e"}
Jul 24 22:53:32.702883 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:32.702832    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cm8cs" event={"ID":"105e164b-d539-4eb1-83e1-0db3b18886ca","Type":"ContainerStarted","Data":"21122e506ce81aa2f5ef32724d8d5793ad607209d4050566bebf3921d9fa28cd"}
Jul 24 22:53:32.705052 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:32.705023    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cmqnc" event={"ID":"eca3657c-182c-47a7-84a7-d82ff5e23942","Type":"ContainerStarted","Data":"bd653510d21859bbcb10a7db5c04289999647c090349bbb1bb5c4b086cc4911b"}
Jul 24 22:53:37.352143 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:37.352099    2568 patch_prober.go:28] interesting pod/image-registry-dcd89596d-77ljd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 22:53:37.352588 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:37.352170    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-dcd89596d-77ljd" podUID="2235fbc2-c533-445c-8a54-8cb7234b9c39" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 22:53:40.738856 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.738819    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x" event={"ID":"97becff6-40a8-4e7a-ab55-0d793f4209d2","Type":"ContainerStarted","Data":"4782a8686d0dceda641d25698aba20a31275c456a72d86c19ca23d2d18fe96f7"}
Jul 24 22:53:40.739369 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.738979    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:40.740491 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.740433    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cmqnc" event={"ID":"eca3657c-182c-47a7-84a7-d82ff5e23942","Type":"ContainerStarted","Data":"f65c030ef0a9264024fb1ee9fd4cd684a2d69956b136c6f5874d05581c58c0b1"}
Jul 24 22:53:40.740623 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.740562    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:53:40.740997 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.740978    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x"
Jul 24 22:53:40.742507 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.742477    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqdf5" event={"ID":"17c261bd-15e5-4c8f-9cf6-8cab547c1c80","Type":"ContainerStarted","Data":"04a60821bc2330b365933540d81a1b92149829524414985d18d385397fc26021"}
Jul 24 22:53:40.744285 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.744263    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cfwc9" event={"ID":"f157f026-514d-4b81-8497-6b39a07b9c58","Type":"ContainerStarted","Data":"4b3f8fa4496f82ad3f3b228fd407374fd7422c9e64c43dd8ae5e90bdf73fbacb"}
Jul 24 22:53:40.744385 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.744311    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cfwc9" event={"ID":"f157f026-514d-4b81-8497-6b39a07b9c58","Type":"ContainerStarted","Data":"c0b720fcf651536f472a534adc3395542ee6c469da71d906ba38c01491e5bb5d"}
Jul 24 22:53:40.744385 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.744334    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:53:40.745930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.745909    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lvlzd" event={"ID":"3cc21e39-0f51-46a7-8104-61530151c730","Type":"ContainerStarted","Data":"127b0658a588afae1a7923041eed9a5539bbfe757ee4f70b134e21cc8f79bd8b"}
Jul 24 22:53:40.746024 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.745948    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lvlzd" event={"ID":"3cc21e39-0f51-46a7-8104-61530151c730","Type":"ContainerStarted","Data":"e952e908420c1f3fb40a8cfd7cf59de11ece62b736e4d1bb0ef469ea090037e7"}
Jul 24 22:53:40.747154 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.747134    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cm8cs" event={"ID":"105e164b-d539-4eb1-83e1-0db3b18886ca","Type":"ContainerStarted","Data":"8148d3307dbab66ceb8ff8326dc017f248a19dab176dfbdc80191505f6aaa27b"}
Jul 24 22:53:40.755971 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.755937    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c69d54cc6-wwt4x" podStartSLOduration=1.471574912 podStartE2EDuration="13.755925628s" totalImagesPullingTime="12.284350716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:40.754729784 +0000 UTC m=+78.031763329" watchObservedRunningTime="2026-07-24 22:53:40.755925628 +0000 UTC m=+78.032959172"
Jul 24 22:53:40.769425 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.769382    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cm8cs" podStartSLOduration=69.443659676 podStartE2EDuration="1m17.769368765s" totalImagesPullingTime="8.325709089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:40.768912587 +0000 UTC m=+78.045946159" watchObservedRunningTime="2026-07-24 22:53:40.769368765 +0000 UTC m=+78.046402357"
Jul 24 22:53:40.788327 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.785277    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cfwc9" podStartSLOduration=34.254058972 podStartE2EDuration="42.785244904s" totalImagesPullingTime="8.531185932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:40.784584331 +0000 UTC m=+78.061617876" watchObservedRunningTime="2026-07-24 22:53:40.785244904 +0000 UTC m=+78.062278446"
Jul 24 22:53:40.824114 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.824061    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cmqnc" podStartSLOduration=69.499736434 podStartE2EDuration="1m17.824044563s" totalImagesPullingTime="8.324308129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:40.823058199 +0000 UTC m=+78.100091742" watchObservedRunningTime="2026-07-24 22:53:40.824044563 +0000 UTC m=+78.101078131"
Jul 24 22:53:40.838528 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.838480    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lvlzd" podStartSLOduration=69.537332503 podStartE2EDuration="1m17.838466005s" totalImagesPullingTime="8.301133502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:40.837555342 +0000 UTC m=+78.114588885" watchObservedRunningTime="2026-07-24 22:53:40.838466005 +0000 UTC m=+78.115499549"
Jul 24 22:53:40.854928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:40.854882    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zqdf5" podStartSLOduration=17.235717861 podStartE2EDuration="34.85486252s" totalImagesPullingTime="17.619144659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:53:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:40.854056035 +0000 UTC m=+78.131089579" watchObservedRunningTime="2026-07-24 22:53:40.85486252 +0000 UTC m=+78.131896063"
Jul 24 22:53:43.106966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.106930    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"]
Jul 24 22:53:43.109070 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.109051    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.112731 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.112709    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 24 22:53:43.112849 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.112710    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 24 22:53:43.112849 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.112772    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vwt6h\""
Jul 24 22:53:43.112849 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.112772    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 22:53:43.117792 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.117775    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 22:53:43.117895 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.117825    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 22:53:43.119776 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.119741    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"]
Jul 24 22:53:43.211999 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.211953    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a121134-ab85-45c1-bf5c-ed6c849e02fc-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.212180 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.212008    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a121134-ab85-45c1-bf5c-ed6c849e02fc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.212180 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.212065    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hvj8\" (UniqueName: \"kubernetes.io/projected/9a121134-ab85-45c1-bf5c-ed6c849e02fc-kube-api-access-9hvj8\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.212180 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.212099    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a121134-ab85-45c1-bf5c-ed6c849e02fc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.312478 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.312448    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a121134-ab85-45c1-bf5c-ed6c849e02fc-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.312478 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.312483    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a121134-ab85-45c1-bf5c-ed6c849e02fc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.312729 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.312505    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hvj8\" (UniqueName: \"kubernetes.io/projected/9a121134-ab85-45c1-bf5c-ed6c849e02fc-kube-api-access-9hvj8\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.312729 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.312524    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a121134-ab85-45c1-bf5c-ed6c849e02fc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.312729 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:43.312626    2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 24 22:53:43.312729 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:43.312677    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a121134-ab85-45c1-bf5c-ed6c849e02fc-prometheus-operator-tls podName:9a121134-ab85-45c1-bf5c-ed6c849e02fc nodeName:}" failed. No retries permitted until 2026-07-24 22:53:43.812663114 +0000 UTC m=+81.089696636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/9a121134-ab85-45c1-bf5c-ed6c849e02fc-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-jvnzt" (UID: "9a121134-ab85-45c1-bf5c-ed6c849e02fc") : secret "prometheus-operator-tls" not found
Jul 24 22:53:43.313153 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.313123    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a121134-ab85-45c1-bf5c-ed6c849e02fc-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.315014 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.314983    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a121134-ab85-45c1-bf5c-ed6c849e02fc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.323767 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.323741    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hvj8\" (UniqueName: \"kubernetes.io/projected/9a121134-ab85-45c1-bf5c-ed6c849e02fc-kube-api-access-9hvj8\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.815905 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.815859    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a121134-ab85-45c1-bf5c-ed6c849e02fc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:43.818223 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:43.818201    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a121134-ab85-45c1-bf5c-ed6c849e02fc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-jvnzt\" (UID: \"9a121134-ab85-45c1-bf5c-ed6c849e02fc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:44.017829 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:44.017778    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"
Jul 24 22:53:44.147568 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:44.147538    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-jvnzt"]
Jul 24 22:53:44.150903 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:44.150879    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a121134_ab85_45c1_bf5c_ed6c849e02fc.slice/crio-bbb88cc98db7d9a26902945914a7ee8c2237dcc6ec3a5f1fdbe2a5a3285fa401 WatchSource:0}: Error finding container bbb88cc98db7d9a26902945914a7ee8c2237dcc6ec3a5f1fdbe2a5a3285fa401: Status 404 returned error can't find the container with id bbb88cc98db7d9a26902945914a7ee8c2237dcc6ec3a5f1fdbe2a5a3285fa401
Jul 24 22:53:44.764762 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:44.764640    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt" event={"ID":"9a121134-ab85-45c1-bf5c-ed6c849e02fc","Type":"ContainerStarted","Data":"bbb88cc98db7d9a26902945914a7ee8c2237dcc6ec3a5f1fdbe2a5a3285fa401"}
Jul 24 22:53:45.770032 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:45.769946    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt" event={"ID":"9a121134-ab85-45c1-bf5c-ed6c849e02fc","Type":"ContainerStarted","Data":"0e55552a3824b5c8a0918eb4db1da098d649235a29c5768ff6f23994e11a6329"}
Jul 24 22:53:45.770032 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:45.770008    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt" event={"ID":"9a121134-ab85-45c1-bf5c-ed6c849e02fc","Type":"ContainerStarted","Data":"72382f8da93c1f8c4c14d6b1a9d26bc76a8551949c948c99b8d7a0b8b22649ae"}
Jul 24 22:53:45.786231 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:45.786172    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-jvnzt" podStartSLOduration=1.488539842 podStartE2EDuration="2.786153941s" totalImagesPullingTime="1.297614099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:45.785873227 +0000 UTC m=+83.062906772" watchObservedRunningTime="2026-07-24 22:53:45.786153941 +0000 UTC m=+83.063187484"
Jul 24 22:53:47.349549 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.349512    2568 patch_prober.go:28] interesting pod/image-registry-dcd89596d-77ljd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 22:53:47.349994 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.349566    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-dcd89596d-77ljd" podUID="2235fbc2-c533-445c-8a54-8cb7234b9c39" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 22:53:47.511900 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.511871    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fxzgx"]
Jul 24 22:53:47.514043 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.514027    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.516476 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.516455    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 22:53:47.516700 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.516672    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 22:53:47.516815 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.516726    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xbwd8\""
Jul 24 22:53:47.516815 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.516757    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 22:53:47.644484 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.644389    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e40a84ee-b980-43f4-9c9a-1b181e5c8519-root\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.644484 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.644435    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cf47\" (UniqueName: \"kubernetes.io/projected/e40a84ee-b980-43f4-9c9a-1b181e5c8519-kube-api-access-8cf47\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.644484 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.644459    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e40a84ee-b980-43f4-9c9a-1b181e5c8519-metrics-client-ca\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.644758 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.644537    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-tls\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.644758 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.644570    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-textfile\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.644758 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.644639    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-wtmp\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.644758 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.644673    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e40a84ee-b980-43f4-9c9a-1b181e5c8519-sys\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.644758 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.644692    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-accelerators-collector-config\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.644758 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.644738    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.738190 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.738158    2568 patch_prober.go:28] interesting pod/image-registry-595bf88bc6-dj7ql container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 22:53:47.738492 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.738220    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql" podUID="5a2b5a24-16bd-4d43-b0e0-6bb5c117359e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 22:53:47.745094 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745071    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-textfile\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745261 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745110    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-wtmp\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745261 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745139    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e40a84ee-b980-43f4-9c9a-1b181e5c8519-sys\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745384 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745282    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-accelerators-collector-config\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745384 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745321    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-wtmp\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745384 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745329    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745510 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745389    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e40a84ee-b980-43f4-9c9a-1b181e5c8519-sys\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745510 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745404    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e40a84ee-b980-43f4-9c9a-1b181e5c8519-root\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745510 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745439    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cf47\" (UniqueName: \"kubernetes.io/projected/e40a84ee-b980-43f4-9c9a-1b181e5c8519-kube-api-access-8cf47\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745510 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745464    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e40a84ee-b980-43f4-9c9a-1b181e5c8519-metrics-client-ca\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745510 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745487    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-textfile\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745689 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745510    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e40a84ee-b980-43f4-9c9a-1b181e5c8519-root\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745689 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745532    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-tls\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745934 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745916    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-accelerators-collector-config\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.745988 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.745974    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e40a84ee-b980-43f4-9c9a-1b181e5c8519-metrics-client-ca\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.747576 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.747558    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.747718 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.747702    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e40a84ee-b980-43f4-9c9a-1b181e5c8519-node-exporter-tls\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.753160 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.753137    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cf47\" (UniqueName: \"kubernetes.io/projected/e40a84ee-b980-43f4-9c9a-1b181e5c8519-kube-api-access-8cf47\") pod \"node-exporter-fxzgx\" (UID: \"e40a84ee-b980-43f4-9c9a-1b181e5c8519\") " pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.823775 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:47.823742    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fxzgx"
Jul 24 22:53:47.834010 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:47.833978    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40a84ee_b980_43f4_9c9a_1b181e5c8519.slice/crio-cb890930546a5387d699e9742f411563df023eae27f1c2c7bbd5b2673be9906e WatchSource:0}: Error finding container cb890930546a5387d699e9742f411563df023eae27f1c2c7bbd5b2673be9906e: Status 404 returned error can't find the container with id cb890930546a5387d699e9742f411563df023eae27f1c2c7bbd5b2673be9906e
Jul 24 22:53:48.783305 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:48.783274    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fxzgx" event={"ID":"e40a84ee-b980-43f4-9c9a-1b181e5c8519","Type":"ContainerStarted","Data":"cb890930546a5387d699e9742f411563df023eae27f1c2c7bbd5b2673be9906e"}
Jul 24 22:53:49.684347 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:49.684313    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-595bf88bc6-dj7ql"
Jul 24 22:53:49.788548 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:49.788508    2568 generic.go:356] "Generic (PLEG): container finished" podID="e40a84ee-b980-43f4-9c9a-1b181e5c8519" containerID="cf46492c0403b08560134b4f18481c0deae0bda5d439de006e92b00c4ca4cdf2" exitCode=0
Jul 24 22:53:49.788932 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:49.788593    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fxzgx" event={"ID":"e40a84ee-b980-43f4-9c9a-1b181e5c8519","Type":"ContainerDied","Data":"cf46492c0403b08560134b4f18481c0deae0bda5d439de006e92b00c4ca4cdf2"}
Jul 24 22:53:50.463711 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.463657    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"]
Jul 24 22:53:50.466029 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.466014    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.468543 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.468521    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 24 22:53:50.468663 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.468568    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qrnpx\""
Jul 24 22:53:50.468663 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.468616    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 24 22:53:50.468840 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.468793    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 24 22:53:50.468945 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.468798    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 24 22:53:50.469462 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.469447    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2unl4t96m4cem\""
Jul 24 22:53:50.469526 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.469495    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 24 22:53:50.478238 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.478217    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"]
Jul 24 22:53:50.569565 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.569536    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.569710 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.569584    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.569710 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.569614    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-tls\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.569710 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.569640    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.569710 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.569661    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fdad54c-a9ea-4f27-a809-06663d90eab0-metrics-client-ca\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.569710 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.569681    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrc24\" (UniqueName: \"kubernetes.io/projected/3fdad54c-a9ea-4f27-a809-06663d90eab0-kube-api-access-zrc24\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.569901 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.569787    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-grpc-tls\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.569901 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.569818    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.670876 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.670840    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrc24\" (UniqueName: \"kubernetes.io/projected/3fdad54c-a9ea-4f27-a809-06663d90eab0-kube-api-access-zrc24\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.671053 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.670887    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-grpc-tls\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.671053 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.670909    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.671053 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.670942    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.671053 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.670974    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.671053 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.670997    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-tls\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.671605 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.671042    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.671605 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.671485    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fdad54c-a9ea-4f27-a809-06663d90eab0-metrics-client-ca\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.672593 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.672406    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fdad54c-a9ea-4f27-a809-06663d90eab0-metrics-client-ca\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.674368 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.674343    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-grpc-tls\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.674669 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.674645    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-tls\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.674768 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.674701    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.678114 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.676644    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.678783 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.678758    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.680904 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.680858    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrc24\" (UniqueName: \"kubernetes.io/projected/3fdad54c-a9ea-4f27-a809-06663d90eab0-kube-api-access-zrc24\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.681870 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.681842    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fdad54c-a9ea-4f27-a809-06663d90eab0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64888c8bd7-6qvn2\" (UID: \"3fdad54c-a9ea-4f27-a809-06663d90eab0\") " pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.753457 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.753375    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cfwc9"
Jul 24 22:53:50.775413 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.775377    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:50.798760 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.796460    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fxzgx" event={"ID":"e40a84ee-b980-43f4-9c9a-1b181e5c8519","Type":"ContainerStarted","Data":"77bd2d0381b73897111d948e51f2363360d7247347d9dd660dbe8f05254d9969"}
Jul 24 22:53:50.798760 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.796574    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fxzgx" event={"ID":"e40a84ee-b980-43f4-9c9a-1b181e5c8519","Type":"ContainerStarted","Data":"3c25510f17dc571794581da3e1a0e06aea4af40fe9e6834911d57323795410b9"}
Jul 24 22:53:50.817503 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.817435    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fxzgx" podStartSLOduration=2.783803899 podStartE2EDuration="3.817414143s" totalImagesPullingTime="970.079588ms" totalInitContainerRuntime="63.530656ms" isStatefulPod=true podCreationTimestamp="2026-07-24 22:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:50.81509291 +0000 UTC m=+88.092126455" watchObservedRunningTime="2026-07-24 22:53:50.817414143 +0000 UTC m=+88.094447692"
Jul 24 22:53:50.916733 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:50.916685    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"]
Jul 24 22:53:50.919191 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:50.919151    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fdad54c_a9ea_4f27_a809_06663d90eab0.slice/crio-1db938029830f6b6c434d2d4165d7fe0fea9b097bcd66da7137e6d2e78c443b9 WatchSource:0}: Error finding container 1db938029830f6b6c434d2d4165d7fe0fea9b097bcd66da7137e6d2e78c443b9: Status 404 returned error can't find the container with id 1db938029830f6b6c434d2d4165d7fe0fea9b097bcd66da7137e6d2e78c443b9
Jul 24 22:53:51.801742 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.801707    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2" event={"ID":"3fdad54c-a9ea-4f27-a809-06663d90eab0","Type":"ContainerStarted","Data":"1db938029830f6b6c434d2d4165d7fe0fea9b097bcd66da7137e6d2e78c443b9"}
Jul 24 22:53:51.877310 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.877278    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"]
Jul 24 22:53:51.879687 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.879664    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:51.883648 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.883627    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 24 22:53:51.883814 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.883651    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 22:53:51.883814 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.883661    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2de50ifqr572q\""
Jul 24 22:53:51.883814 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.883758    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 24 22:53:51.884023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.883931    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 24 22:53:51.884023 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.884012    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zfw5w\""
Jul 24 22:53:51.891536 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.891503    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"]
Jul 24 22:53:51.981465 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.981432    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-metrics-server-audit-profiles\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:51.981465 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.981470    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:51.981719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.981508    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-secret-metrics-server-tls\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:51.981719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.981533    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-secret-metrics-server-client-certs\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:51.981719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.981634    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-audit-log\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:51.981719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.981661    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-client-ca-bundle\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:51.981719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:51.981680    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zlhx\" (UniqueName: \"kubernetes.io/projected/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-kube-api-access-8zlhx\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.083092 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.083004    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-audit-log\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.083092 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.083041    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-client-ca-bundle\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.083092 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.083062    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zlhx\" (UniqueName: \"kubernetes.io/projected/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-kube-api-access-8zlhx\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.083092 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.083089    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-metrics-server-audit-profiles\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.083443 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.083113    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.083443 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.083164    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-secret-metrics-server-tls\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.083443 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.083197    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-secret-metrics-server-client-certs\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.083591 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.083512    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-audit-log\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.083990 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.083970    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.084496 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.084473    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-metrics-server-audit-profiles\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.085809 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.085783    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-secret-metrics-server-client-certs\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.085874 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.085855    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-secret-metrics-server-tls\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.085911 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.085891    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-client-ca-bundle\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.091308 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.091271    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zlhx\" (UniqueName: \"kubernetes.io/projected/03ba85c6-1163-4ee1-aa37-ac3c6e3493b5-kube-api-access-8zlhx\") pod \"metrics-server-7cfb7c5c67-6b7wk\" (UID: \"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5\") " pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.192088 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.192054    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:53:52.245443 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.243655    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"]
Jul 24 22:53:52.246325 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.246309    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:52.248859 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.248837    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 24 22:53:52.248985 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.248907    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rlwl4\""
Jul 24 22:53:52.262035 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.261998    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"]
Jul 24 22:53:52.336050 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.335975    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"]
Jul 24 22:53:52.340119 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:52.340081    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03ba85c6_1163_4ee1_aa37_ac3c6e3493b5.slice/crio-20c0f3497012d10d2f89325db13db59748a57f99abf46ee80b12fcf7f50b9d32 WatchSource:0}: Error finding container 20c0f3497012d10d2f89325db13db59748a57f99abf46ee80b12fcf7f50b9d32: Status 404 returned error can't find the container with id 20c0f3497012d10d2f89325db13db59748a57f99abf46ee80b12fcf7f50b9d32
Jul 24 22:53:52.364673 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.364630    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-dcd89596d-77ljd" podUID="2235fbc2-c533-445c-8a54-8cb7234b9c39" containerName="registry" containerID="cri-o://74e789771dcb41f7f7e8f43fc186aa41be6c54e8a85872167c8d9ccb1a355490" gracePeriod=30
Jul 24 22:53:52.387010 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.386981    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a665f6-1d07-498a-b1ed-c91e6ba7700d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-fw6st\" (UID: \"02a665f6-1d07-498a-b1ed-c91e6ba7700d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:52.387153 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.387062    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4q2m\" (UniqueName: \"kubernetes.io/projected/02a665f6-1d07-498a-b1ed-c91e6ba7700d-kube-api-access-p4q2m\") pod \"monitoring-plugin-68b7fc9574-fw6st\" (UID: \"02a665f6-1d07-498a-b1ed-c91e6ba7700d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:52.488488 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.488457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4q2m\" (UniqueName: \"kubernetes.io/projected/02a665f6-1d07-498a-b1ed-c91e6ba7700d-kube-api-access-p4q2m\") pod \"monitoring-plugin-68b7fc9574-fw6st\" (UID: \"02a665f6-1d07-498a-b1ed-c91e6ba7700d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:52.488619 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.488521    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a665f6-1d07-498a-b1ed-c91e6ba7700d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-fw6st\" (UID: \"02a665f6-1d07-498a-b1ed-c91e6ba7700d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:52.488699 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:52.488676    2568 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 24 22:53:52.488818 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:53:52.488751    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a665f6-1d07-498a-b1ed-c91e6ba7700d-monitoring-plugin-cert podName:02a665f6-1d07-498a-b1ed-c91e6ba7700d nodeName:}" failed. No retries permitted until 2026-07-24 22:53:52.988729668 +0000 UTC m=+90.265763207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/02a665f6-1d07-498a-b1ed-c91e6ba7700d-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-fw6st" (UID: "02a665f6-1d07-498a-b1ed-c91e6ba7700d") : secret "monitoring-plugin-cert" not found
Jul 24 22:53:52.497353 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.497321    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4q2m\" (UniqueName: \"kubernetes.io/projected/02a665f6-1d07-498a-b1ed-c91e6ba7700d-kube-api-access-p4q2m\") pod \"monitoring-plugin-68b7fc9574-fw6st\" (UID: \"02a665f6-1d07-498a-b1ed-c91e6ba7700d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:52.697832 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.697804    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7465458b77-pxq84"]
Jul 24 22:53:52.703659 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.703637    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.706089 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.706062    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 24 22:53:52.706276 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.706238    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 24 22:53:52.706362 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.706306    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 24 22:53:52.706414 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.706063    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 24 22:53:52.706707 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.706684    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-cqtgc\""
Jul 24 22:53:52.706802 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.706744    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 24 22:53:52.711910 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.711887    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 24 22:53:52.714325 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.714304    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7465458b77-pxq84"]
Jul 24 22:53:52.792678 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.792641    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-telemeter-client-tls\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.792848 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.792688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-metrics-client-ca\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.792848 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.792781    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-secret-telemeter-client\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.792848 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.792832    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrwtk\" (UniqueName: \"kubernetes.io/projected/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-kube-api-access-xrwtk\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.792993 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.792876    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-serving-certs-ca-bundle\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.792993 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.792896    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-federate-client-tls\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.792993 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.792936    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.792993 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.792987    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.807692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.807644    2568 generic.go:356] "Generic (PLEG): container finished" podID="2235fbc2-c533-445c-8a54-8cb7234b9c39" containerID="74e789771dcb41f7f7e8f43fc186aa41be6c54e8a85872167c8d9ccb1a355490" exitCode=0
Jul 24 22:53:52.808116 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.807737    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dcd89596d-77ljd" event={"ID":"2235fbc2-c533-445c-8a54-8cb7234b9c39","Type":"ContainerDied","Data":"74e789771dcb41f7f7e8f43fc186aa41be6c54e8a85872167c8d9ccb1a355490"}
Jul 24 22:53:52.809096 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.809055    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk" event={"ID":"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5","Type":"ContainerStarted","Data":"20c0f3497012d10d2f89325db13db59748a57f99abf46ee80b12fcf7f50b9d32"}
Jul 24 22:53:52.894027 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.893980    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.894217 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.894059    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.894217 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.894082    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-telemeter-client-tls\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.894217 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.894101    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-metrics-client-ca\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.894217 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.894133    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-secret-telemeter-client\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.894217 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.894169    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrwtk\" (UniqueName: \"kubernetes.io/projected/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-kube-api-access-xrwtk\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.894217 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.894189    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-serving-certs-ca-bundle\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.894217 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.894205    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-federate-client-tls\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.895139 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.895111    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-metrics-client-ca\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.895273 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.895176    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.895352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.895283    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-serving-certs-ca-bundle\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.897153 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.897132    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-federate-client-tls\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.897336 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.897315    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.897426 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.897340    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-secret-telemeter-client\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.897638 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.897612    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-telemeter-client-tls\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.910660 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.910636    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrwtk\" (UniqueName: \"kubernetes.io/projected/d4e4cd8b-42a7-4054-8b07-8ed4a091db70-kube-api-access-xrwtk\") pod \"telemeter-client-7465458b77-pxq84\" (UID: \"d4e4cd8b-42a7-4054-8b07-8ed4a091db70\") " pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:52.995711 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.995622    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a665f6-1d07-498a-b1ed-c91e6ba7700d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-fw6st\" (UID: \"02a665f6-1d07-498a-b1ed-c91e6ba7700d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:52.998267 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:52.998232    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a665f6-1d07-498a-b1ed-c91e6ba7700d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-fw6st\" (UID: \"02a665f6-1d07-498a-b1ed-c91e6ba7700d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:53.015497 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.015450    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7465458b77-pxq84"
Jul 24 22:53:53.080835 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.080812    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:53.165402 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.165377    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:53.197296 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.197265    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2235fbc2-c533-445c-8a54-8cb7234b9c39-ca-trust-extracted\") pod \"2235fbc2-c533-445c-8a54-8cb7234b9c39\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") "
Jul 24 22:53:53.197445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.197314    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-image-registry-private-configuration\") pod \"2235fbc2-c533-445c-8a54-8cb7234b9c39\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") "
Jul 24 22:53:53.197445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.197354    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls\") pod \"2235fbc2-c533-445c-8a54-8cb7234b9c39\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") "
Jul 24 22:53:53.197445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.197393    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8s9s\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-kube-api-access-f8s9s\") pod \"2235fbc2-c533-445c-8a54-8cb7234b9c39\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") "
Jul 24 22:53:53.197445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.197429    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-trusted-ca\") pod \"2235fbc2-c533-445c-8a54-8cb7234b9c39\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") "
Jul 24 22:53:53.197644 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.197457    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-bound-sa-token\") pod \"2235fbc2-c533-445c-8a54-8cb7234b9c39\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") "
Jul 24 22:53:53.197644 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.197490    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-certificates\") pod \"2235fbc2-c533-445c-8a54-8cb7234b9c39\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") "
Jul 24 22:53:53.197644 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.197510    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-installation-pull-secrets\") pod \"2235fbc2-c533-445c-8a54-8cb7234b9c39\" (UID: \"2235fbc2-c533-445c-8a54-8cb7234b9c39\") "
Jul 24 22:53:53.198126 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.198088    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2235fbc2-c533-445c-8a54-8cb7234b9c39" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:53:53.198475 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.198451    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2235fbc2-c533-445c-8a54-8cb7234b9c39" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:53:53.200800 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.200772    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2235fbc2-c533-445c-8a54-8cb7234b9c39" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:53:53.200904 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.200793    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2235fbc2-c533-445c-8a54-8cb7234b9c39" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:53:53.200965 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.200905    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2235fbc2-c533-445c-8a54-8cb7234b9c39" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:53:53.201153 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.201134    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-kube-api-access-f8s9s" (OuterVolumeSpecName: "kube-api-access-f8s9s") pod "2235fbc2-c533-445c-8a54-8cb7234b9c39" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39"). InnerVolumeSpecName "kube-api-access-f8s9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:53:53.201454 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.201433    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2235fbc2-c533-445c-8a54-8cb7234b9c39" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:53:53.207005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.206979    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2235fbc2-c533-445c-8a54-8cb7234b9c39-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2235fbc2-c533-445c-8a54-8cb7234b9c39" (UID: "2235fbc2-c533-445c-8a54-8cb7234b9c39"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:53:53.210361 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.210334    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7465458b77-pxq84"]
Jul 24 22:53:53.212578 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:53.212552    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e4cd8b_42a7_4054_8b07_8ed4a091db70.slice/crio-2481ee904a8dfed034026c437e6ad159c8fcf349d2afe22c04222814e818bfb4 WatchSource:0}: Error finding container 2481ee904a8dfed034026c437e6ad159c8fcf349d2afe22c04222814e818bfb4: Status 404 returned error can't find the container with id 2481ee904a8dfed034026c437e6ad159c8fcf349d2afe22c04222814e818bfb4
Jul 24 22:53:53.299038 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.299014    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8s9s\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-kube-api-access-f8s9s\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:53:53.299038 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.299038    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-trusted-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:53:53.299038 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.299047    2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-bound-sa-token\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:53:53.299244 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.299056    2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-certificates\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:53:53.299244 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.299064    2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-installation-pull-secrets\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:53:53.299244 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.299073    2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2235fbc2-c533-445c-8a54-8cb7234b9c39-ca-trust-extracted\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:53:53.299244 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.299081    2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2235fbc2-c533-445c-8a54-8cb7234b9c39-image-registry-private-configuration\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:53:53.299244 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.299090    2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2235fbc2-c533-445c-8a54-8cb7234b9c39-registry-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:53:53.313699 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.313672    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"]
Jul 24 22:53:53.317435 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:53.317402    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a665f6_1d07_498a_b1ed_c91e6ba7700d.slice/crio-54ba6af9ed84851edd59c0d658ebae223aeb159d131460cf49281ede8e4f9970 WatchSource:0}: Error finding container 54ba6af9ed84851edd59c0d658ebae223aeb159d131460cf49281ede8e4f9970: Status 404 returned error can't find the container with id 54ba6af9ed84851edd59c0d658ebae223aeb159d131460cf49281ede8e4f9970
Jul 24 22:53:53.774507 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.772933    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 22:53:53.774507 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.773399    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2235fbc2-c533-445c-8a54-8cb7234b9c39" containerName="registry"
Jul 24 22:53:53.774507 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.773414    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2235fbc2-c533-445c-8a54-8cb7234b9c39" containerName="registry"
Jul 24 22:53:53.774507 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.773491    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2235fbc2-c533-445c-8a54-8cb7234b9c39" containerName="registry"
Jul 24 22:53:53.784476 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.784445    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.788232 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.788173    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 22:53:53.788390 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.788342    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 22:53:53.788390 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.788370    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 22:53:53.788578 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.788553    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 22:53:53.788685 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.788579    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 22:53:53.788685 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.788633    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 22:53:53.788685 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.788645    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 22:53:53.788857 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.788841    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 22:53:53.788909 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.788891    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 22:53:53.789101 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.789085    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fs4nelmbibmpl\""
Jul 24 22:53:53.789189 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.789121    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 22:53:53.789442 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.789423    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 22:53:53.789992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.789973    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-79xf6\""
Jul 24 22:53:53.791872 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.791852    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 22:53:53.805275 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.796433    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 22:53:53.805275 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.801148    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 22:53:53.805275 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.803536    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 22:53:53.818971 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.818866    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dcd89596d-77ljd" event={"ID":"2235fbc2-c533-445c-8a54-8cb7234b9c39","Type":"ContainerDied","Data":"546462938a456a422254ae6ae34755100194912453cdfbc05770b5cf229b6955"}
Jul 24 22:53:53.820324 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.819496    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dcd89596d-77ljd"
Jul 24 22:53:53.820575 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.820459    2568 scope.go:117] "RemoveContainer" containerID="74e789771dcb41f7f7e8f43fc186aa41be6c54e8a85872167c8d9ccb1a355490"
Jul 24 22:53:53.830821 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.830766    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2" event={"ID":"3fdad54c-a9ea-4f27-a809-06663d90eab0","Type":"ContainerStarted","Data":"f5a2da5b72413f56620ca3620a09dc17ad78e14a9755e5ee87cc4768cccc1c9b"}
Jul 24 22:53:53.830930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.830839    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2" event={"ID":"3fdad54c-a9ea-4f27-a809-06663d90eab0","Type":"ContainerStarted","Data":"81884aa36dad98f868dc097849c7464c8eb2c0d36f3df3d54744fb4ed2b31ece"}
Jul 24 22:53:53.830930 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.830881    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2" event={"ID":"3fdad54c-a9ea-4f27-a809-06663d90eab0","Type":"ContainerStarted","Data":"f1bd96d024a278c22ac604878da88245a33f09ef0af3782e461f679302552e21"}
Jul 24 22:53:53.833320 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.833289    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st" event={"ID":"02a665f6-1d07-498a-b1ed-c91e6ba7700d","Type":"ContainerStarted","Data":"54ba6af9ed84851edd59c0d658ebae223aeb159d131460cf49281ede8e4f9970"}
Jul 24 22:53:53.834999 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.834975    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7465458b77-pxq84" event={"ID":"d4e4cd8b-42a7-4054-8b07-8ed4a091db70","Type":"ContainerStarted","Data":"2481ee904a8dfed034026c437e6ad159c8fcf349d2afe22c04222814e818bfb4"}
Jul 24 22:53:53.842460 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.842438    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dcd89596d-77ljd"]
Jul 24 22:53:53.851263 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.851106    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-dcd89596d-77ljd"]
Jul 24 22:53:53.903378 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903343    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903575 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903394    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903575 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903420    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903575 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903442    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-web-config\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903575 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903467    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903575 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903488    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903575 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903511    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903575 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903539    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903587    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903630    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903656    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config-out\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903680    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903718    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7knsv\" (UniqueName: \"kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-kube-api-access-7knsv\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903770    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903797    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903845    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903874    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.903923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903900    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.904379 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903927    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:53.904379 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:53.903966    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005309    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005383    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005426    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005453    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config-out\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005477    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005504    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7knsv\" (UniqueName: \"kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-kube-api-access-7knsv\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005545    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005572    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005606    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005632    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005654    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005681    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005712    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005748    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005783    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005808    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.008966 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005833    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-web-config\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.009928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005859    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.009928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005882    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.009928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.005908    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.009928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.006966    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.009928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.007228    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.009928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.008145    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.009928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.008778    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.009928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.009072    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.009928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.009659    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.011067 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.010183    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.015266 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.012794    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.015266 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.014897    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config-out\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.015433 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.015337    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.015733 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.015675    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.016183 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.016150    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.016612 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.016591    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.017220 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.017149    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-web-config\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.017338 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.017222    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.017935 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.017886    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.018182 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.018149    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.019421 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.019382    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.021696 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.021651    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.025892 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.025816    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7knsv\" (UniqueName: \"kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-kube-api-access-7knsv\") pod \"prometheus-k8s-0\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:54.116954 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:54.116909    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:53:55.257486 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.257451    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2235fbc2-c533-445c-8a54-8cb7234b9c39" path="/var/lib/kubelet/pods/2235fbc2-c533-445c-8a54-8cb7234b9c39/volumes"
Jul 24 22:53:55.667559 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.667527    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 22:53:55.669713 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:53:55.669682    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb03511c_4e59_4ebb_9e12_2aa51c469ca2.slice/crio-eecf5bfdd470da351a3112d194e09ccdd53336d916db66e7df8645e013843d4c WatchSource:0}: Error finding container eecf5bfdd470da351a3112d194e09ccdd53336d916db66e7df8645e013843d4c: Status 404 returned error can't find the container with id eecf5bfdd470da351a3112d194e09ccdd53336d916db66e7df8645e013843d4c
Jul 24 22:53:55.845977 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.845939    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk" event={"ID":"03ba85c6-1163-4ee1-aa37-ac3c6e3493b5","Type":"ContainerStarted","Data":"301e46adb4e9f9584dcb3a5cd3dbc53e2d179a3be94357b31a3d8fbdae8ecb47"}
Jul 24 22:53:55.847222 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.847190    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerStarted","Data":"eecf5bfdd470da351a3112d194e09ccdd53336d916db66e7df8645e013843d4c"}
Jul 24 22:53:55.850361 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.850335    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2" event={"ID":"3fdad54c-a9ea-4f27-a809-06663d90eab0","Type":"ContainerStarted","Data":"4d7e89cf56d7cd52632f26512b1112a13b8d72e11ea5760ce7fe062b3b0bc56e"}
Jul 24 22:53:55.850502 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.850398    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2" event={"ID":"3fdad54c-a9ea-4f27-a809-06663d90eab0","Type":"ContainerStarted","Data":"ff2e3395f824edd8bef2303aa8719d72ef22fc2d83de5fbe7eec5732ddc4ac2d"}
Jul 24 22:53:55.850502 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.850440    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2" event={"ID":"3fdad54c-a9ea-4f27-a809-06663d90eab0","Type":"ContainerStarted","Data":"398e50d00d27a31d7b67f2cf3ff21f43397fd0a989c530ea27aef1eced5e00dd"}
Jul 24 22:53:55.850502 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.850486    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:53:55.851863 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.851843    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st" event={"ID":"02a665f6-1d07-498a-b1ed-c91e6ba7700d","Type":"ContainerStarted","Data":"3e26b558fcf3a2599b53e4fa23aec787a41dd4659312aac3e7a1b2cfbdf304a6"}
Jul 24 22:53:55.851975 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.851953    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:55.853400 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.853374    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7465458b77-pxq84" event={"ID":"d4e4cd8b-42a7-4054-8b07-8ed4a091db70","Type":"ContainerStarted","Data":"0ead04be6b1f640cee779384fabd81a690615295d5f4bc5b5f3ef32f4a6369da"}
Jul 24 22:53:55.857010 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.856990    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st"
Jul 24 22:53:55.862522 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.862482    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk" podStartSLOduration=1.697673381 podStartE2EDuration="4.862468126s" totalImagesPullingTime="3.164794745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:55.862020254 +0000 UTC m=+93.139053799" watchObservedRunningTime="2026-07-24 22:53:55.862468126 +0000 UTC m=+93.139501667"
Jul 24 22:53:55.882402 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.882341    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-fw6st" podStartSLOduration=1.6925307969999999 podStartE2EDuration="3.882319475s" totalImagesPullingTime="2.189788678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:55.880730792 +0000 UTC m=+93.157764340" watchObservedRunningTime="2026-07-24 22:53:55.882319475 +0000 UTC m=+93.159353020"
Jul 24 22:53:55.902621 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:55.902570    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2" podStartSLOduration=1.554154806 podStartE2EDuration="5.902551265s" totalImagesPullingTime="4.348396459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:55.902075517 +0000 UTC m=+93.179109092" watchObservedRunningTime="2026-07-24 22:53:55.902551265 +0000 UTC m=+93.179584810"
Jul 24 22:53:56.858440 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:56.858415    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerID="7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f" exitCode=0
Jul 24 22:53:56.858746 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:56.858457    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerDied","Data":"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"}
Jul 24 22:53:56.860583 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:56.860554    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7465458b77-pxq84" event={"ID":"d4e4cd8b-42a7-4054-8b07-8ed4a091db70","Type":"ContainerStarted","Data":"6f06e002222587b069c28121b8f9f687deda37f1fa34716471018cbc87f2c418"}
Jul 24 22:53:57.873100 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:57.873011    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7465458b77-pxq84" event={"ID":"d4e4cd8b-42a7-4054-8b07-8ed4a091db70","Type":"ContainerStarted","Data":"3be1750fadc8f2d7d7b64c2c7bb66c3e184cf4cbf641f80a0e7992b858a9215b"}
Jul 24 22:53:57.896721 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:53:57.896667    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7465458b77-pxq84" podStartSLOduration=2.485040257 podStartE2EDuration="5.896650582s" totalImagesPullingTime="3.411610325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:53:57.894056541 +0000 UTC m=+95.171090100" watchObservedRunningTime="2026-07-24 22:53:57.896650582 +0000 UTC m=+95.173684125"
Jul 24 22:54:00.890295 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:00.890265    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerStarted","Data":"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"}
Jul 24 22:54:00.890637 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:00.890335    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerStarted","Data":"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"}
Jul 24 22:54:00.890637 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:00.890361    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerStarted","Data":"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"}
Jul 24 22:54:00.890637 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:00.890384    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerStarted","Data":"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"}
Jul 24 22:54:00.890637 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:00.890408    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerStarted","Data":"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"}
Jul 24 22:54:01.867525 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:01.867184    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-64888c8bd7-6qvn2"
Jul 24 22:54:01.902756 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:01.902707    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerStarted","Data":"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"}
Jul 24 22:54:01.936590 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:01.936537    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.41340004 podStartE2EDuration="8.936519514s" totalImagesPullingTime="4.500320369s" totalInitContainerRuntime="22.799105ms" isStatefulPod=false podCreationTimestamp="2026-07-24 22:53:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:54:01.934967721 +0000 UTC m=+99.212001294" watchObservedRunningTime="2026-07-24 22:54:01.936519514 +0000 UTC m=+99.213553057"
Jul 24 22:54:04.119524 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:04.119491    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:54:11.753278 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:11.753227    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cmqnc"
Jul 24 22:54:12.193155 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:12.193126    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:54:12.193481 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:12.193466    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:54:20.985828 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:20.985794    2568 generic.go:356] "Generic (PLEG): container finished" podID="0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5" containerID="bdee97fe1070628597684e5db9b8a8de6ceaf3234f1c1b680600324def863632" exitCode=0
Jul 24 22:54:20.986218 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:20.985866    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv" event={"ID":"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5","Type":"ContainerDied","Data":"bdee97fe1070628597684e5db9b8a8de6ceaf3234f1c1b680600324def863632"}
Jul 24 22:54:20.986290 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:20.986238    2568 scope.go:117] "RemoveContainer" containerID="bdee97fe1070628597684e5db9b8a8de6ceaf3234f1c1b680600324def863632"
Jul 24 22:54:21.992080 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:21.992048    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-wwwqv" event={"ID":"0a2a3cbd-ad24-4d6f-80b9-5ba68331bfa5","Type":"ContainerStarted","Data":"c8a948d0e84a43fd23183c863b36bc6ee475018d24632165b6c2f34554c865eb"}
Jul 24 22:54:31.028379 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:31.028346    2568 generic.go:356] "Generic (PLEG): container finished" podID="9e7719c2-57e4-426d-8aee-742e2e535b19" containerID="12d5305c9fe387d29b91090cb9807fcd9df282b41dc41ce6d07ea53041d8c516" exitCode=0
Jul 24 22:54:31.028748 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:31.028412    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m78lj" event={"ID":"9e7719c2-57e4-426d-8aee-742e2e535b19","Type":"ContainerDied","Data":"12d5305c9fe387d29b91090cb9807fcd9df282b41dc41ce6d07ea53041d8c516"}
Jul 24 22:54:31.028748 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:31.028746    2568 scope.go:117] "RemoveContainer" containerID="12d5305c9fe387d29b91090cb9807fcd9df282b41dc41ce6d07ea53041d8c516"
Jul 24 22:54:32.033670 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:32.033635    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m78lj" event={"ID":"9e7719c2-57e4-426d-8aee-742e2e535b19","Type":"ContainerStarted","Data":"11aab6bbc039a44255a266951e01ee9869192dbc091b450a758fb785800fdf60"}
Jul 24 22:54:32.197903 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:32.197872    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:54:32.201982 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:32.201960    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7cfb7c5c67-6b7wk"
Jul 24 22:54:54.117172 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:54.117134    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:54:54.136527 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:54.136500    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:54:55.138317 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:54:55.138288    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:12.121210 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.121170    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 22:55:12.121644 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.121479    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="prometheus" containerID="cri-o://f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa" gracePeriod=600
Jul 24 22:55:12.121644 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.121547    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6" gracePeriod=600
Jul 24 22:55:12.121644 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.121552    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="thanos-sidecar" containerID="cri-o://105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49" gracePeriod=600
Jul 24 22:55:12.121644 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.121579    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy-web" containerID="cri-o://7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c" gracePeriod=600
Jul 24 22:55:12.121644 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.121576    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="config-reloader" containerID="cri-o://0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6" gracePeriod=600
Jul 24 22:55:12.121878 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.121555    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy" containerID="cri-o://e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193" gracePeriod=600
Jul 24 22:55:12.389263 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.389226    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:12.515157 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515123    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7knsv\" (UniqueName: \"kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-kube-api-access-7knsv\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515157 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515170    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-thanos-prometheus-http-client-file\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515192    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-tls\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515219    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-0\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515245    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-serving-certs-ca-bundle\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515303    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515341    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-2\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515366    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-kube-rbac-proxy\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515395    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515423    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515445 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515449    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-db\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515894 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515479    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-kubelet-serving-ca-bundle\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515894 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515519    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config-out\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515894 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515560    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-web-config\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515894 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515595    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-1\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515894 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515619    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-metrics-client-ca\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515894 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515658    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-trusted-ca-bundle\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515894 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515681    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-grpc-tls\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515894 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515709    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:55:12.515894 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515723    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-metrics-client-certs\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.515894 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.515793    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-tls-assets\") pod \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\" (UID: \"cb03511c-4e59-4ebb-9e12-2aa51c469ca2\") "
Jul 24 22:55:12.516403 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.516075    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.516403 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.516222    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:55:12.516808 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.516781    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:55:12.516959 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.516855    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:55:12.518085 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.518057    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:12.518192 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.518080    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:12.518192 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.518121    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:12.518655 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.518327    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-kube-api-access-7knsv" (OuterVolumeSpecName: "kube-api-access-7knsv") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "kube-api-access-7knsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:55:12.518655 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.518545    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:12.518776 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.518652    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:55:12.518776 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.518763    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:55:12.519036 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.519010    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:55:12.519227 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.519199    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config-out" (OuterVolumeSpecName: "config-out") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:55:12.519358 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.519337    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 22:55:12.519680 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.519656    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:55:12.520239 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.520214    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config" (OuterVolumeSpecName: "config") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:12.520383 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.520294    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:12.520748 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.520721    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:12.520933 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.520915    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:12.530277 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.530243    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-web-config" (OuterVolumeSpecName: "web-config") pod "cb03511c-4e59-4ebb-9e12-2aa51c469ca2" (UID: "cb03511c-4e59-4ebb-9e12-2aa51c469ca2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 22:55:12.617088 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617051    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617088 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617081    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-metrics-client-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617088 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617093    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617102    2568 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-grpc-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617113    2568 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-metrics-client-certs\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617122    2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-tls-assets\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617130    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7knsv\" (UniqueName: \"kubernetes.io/projected/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-kube-api-access-7knsv\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617139    2568 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617149    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617156    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617167    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617175    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617183    2568 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-kube-rbac-proxy\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617193    2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617201    2568 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617208    2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-prometheus-k8s-db\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617215    2568 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617223    2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-config-out\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:12.617350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:12.617231    2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cb03511c-4e59-4ebb-9e12-2aa51c469ca2-web-config\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:55:13.202850 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.202811    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerID="7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6" exitCode=0
Jul 24 22:55:13.202850 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.202843    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerID="e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193" exitCode=0
Jul 24 22:55:13.202850 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.202854    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerID="7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c" exitCode=0
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.202866    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerID="105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49" exitCode=0
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.202873    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerID="0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6" exitCode=0
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.202882    2568 generic.go:356] "Generic (PLEG): container finished" podID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerID="f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa" exitCode=0
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.202893    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerDied","Data":"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"}
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.202910    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.202998    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerDied","Data":"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"}
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.203027    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerDied","Data":"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"}
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.203052    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerDied","Data":"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"}
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.203070    2568 scope.go:117] "RemoveContainer" containerID="7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.203088    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerDied","Data":"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"}
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.203114    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerDied","Data":"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"}
Jul 24 22:55:13.203352 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.203137    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cb03511c-4e59-4ebb-9e12-2aa51c469ca2","Type":"ContainerDied","Data":"eecf5bfdd470da351a3112d194e09ccdd53336d916db66e7df8645e013843d4c"}
Jul 24 22:55:13.211553 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.211513    2568 scope.go:117] "RemoveContainer" containerID="e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"
Jul 24 22:55:13.219124 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.219106    2568 scope.go:117] "RemoveContainer" containerID="7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"
Jul 24 22:55:13.225727 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.225708    2568 scope.go:117] "RemoveContainer" containerID="105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"
Jul 24 22:55:13.229456 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.229431    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 22:55:13.233963 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.233947    2568 scope.go:117] "RemoveContainer" containerID="0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"
Jul 24 22:55:13.234519 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.234497    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 22:55:13.240927 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.240906    2568 scope.go:117] "RemoveContainer" containerID="f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"
Jul 24 22:55:13.248047 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.248030    2568 scope.go:117] "RemoveContainer" containerID="7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"
Jul 24 22:55:13.255533 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.255518    2568 scope.go:117] "RemoveContainer" containerID="7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"
Jul 24 22:55:13.255810 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:55:13.255789    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": container with ID starting with 7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6 not found: ID does not exist" containerID="7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"
Jul 24 22:55:13.255878 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.255820    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"} err="failed to get container status \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": rpc error: code = NotFound desc = could not find container \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": container with ID starting with 7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6 not found: ID does not exist"
Jul 24 22:55:13.255878 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.255856    2568 scope.go:117] "RemoveContainer" containerID="e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"
Jul 24 22:55:13.256108 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:55:13.256093    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": container with ID starting with e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193 not found: ID does not exist" containerID="e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"
Jul 24 22:55:13.256158 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.256115    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"} err="failed to get container status \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": rpc error: code = NotFound desc = could not find container \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": container with ID starting with e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193 not found: ID does not exist"
Jul 24 22:55:13.256158 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.256126    2568 scope.go:117] "RemoveContainer" containerID="7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"
Jul 24 22:55:13.256319 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:55:13.256305    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": container with ID starting with 7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c not found: ID does not exist" containerID="7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"
Jul 24 22:55:13.256373 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.256324    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"} err="failed to get container status \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": rpc error: code = NotFound desc = could not find container \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": container with ID starting with 7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c not found: ID does not exist"
Jul 24 22:55:13.256373 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.256339    2568 scope.go:117] "RemoveContainer" containerID="105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"
Jul 24 22:55:13.256575 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:55:13.256556    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": container with ID starting with 105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49 not found: ID does not exist" containerID="105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"
Jul 24 22:55:13.256634 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.256583    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"} err="failed to get container status \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": rpc error: code = NotFound desc = could not find container \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": container with ID starting with 105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49 not found: ID does not exist"
Jul 24 22:55:13.256634 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.256605    2568 scope.go:117] "RemoveContainer" containerID="0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"
Jul 24 22:55:13.256880 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:55:13.256864    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": container with ID starting with 0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6 not found: ID does not exist" containerID="0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"
Jul 24 22:55:13.256939 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.256884    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"} err="failed to get container status \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": rpc error: code = NotFound desc = could not find container \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": container with ID starting with 0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6 not found: ID does not exist"
Jul 24 22:55:13.256939 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.256906    2568 scope.go:117] "RemoveContainer" containerID="f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"
Jul 24 22:55:13.257040 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.256990    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" path="/var/lib/kubelet/pods/cb03511c-4e59-4ebb-9e12-2aa51c469ca2/volumes"
Jul 24 22:55:13.257204 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:55:13.257185    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": container with ID starting with f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa not found: ID does not exist" containerID="f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"
Jul 24 22:55:13.257204 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.257201    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"} err="failed to get container status \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": rpc error: code = NotFound desc = could not find container \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": container with ID starting with f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa not found: ID does not exist"
Jul 24 22:55:13.257566 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.257215    2568 scope.go:117] "RemoveContainer" containerID="7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"
Jul 24 22:55:13.257979 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:55:13.257619    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": container with ID starting with 7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f not found: ID does not exist" containerID="7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"
Jul 24 22:55:13.257979 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.257650    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"} err="failed to get container status \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": rpc error: code = NotFound desc = could not find container \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": container with ID starting with 7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f not found: ID does not exist"
Jul 24 22:55:13.257979 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.257673    2568 scope.go:117] "RemoveContainer" containerID="7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"
Jul 24 22:55:13.257979 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.257932    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"} err="failed to get container status \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": rpc error: code = NotFound desc = could not find container \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": container with ID starting with 7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6 not found: ID does not exist"
Jul 24 22:55:13.257979 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.257951    2568 scope.go:117] "RemoveContainer" containerID="e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"
Jul 24 22:55:13.258278 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.258182    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"} err="failed to get container status \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": rpc error: code = NotFound desc = could not find container \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": container with ID starting with e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193 not found: ID does not exist"
Jul 24 22:55:13.258278 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.258201    2568 scope.go:117] "RemoveContainer" containerID="7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"
Jul 24 22:55:13.258513 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.258492    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"} err="failed to get container status \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": rpc error: code = NotFound desc = could not find container \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": container with ID starting with 7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c not found: ID does not exist"
Jul 24 22:55:13.258567 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.258514    2568 scope.go:117] "RemoveContainer" containerID="105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"
Jul 24 22:55:13.258759 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.258744    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"} err="failed to get container status \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": rpc error: code = NotFound desc = could not find container \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": container with ID starting with 105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49 not found: ID does not exist"
Jul 24 22:55:13.258815 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.258760    2568 scope.go:117] "RemoveContainer" containerID="0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"
Jul 24 22:55:13.258975 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.258953    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"} err="failed to get container status \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": rpc error: code = NotFound desc = could not find container \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": container with ID starting with 0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6 not found: ID does not exist"
Jul 24 22:55:13.259027 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.258978    2568 scope.go:117] "RemoveContainer" containerID="f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"
Jul 24 22:55:13.259221 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259200    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"} err="failed to get container status \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": rpc error: code = NotFound desc = could not find container \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": container with ID starting with f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa not found: ID does not exist"
Jul 24 22:55:13.259315 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259223    2568 scope.go:117] "RemoveContainer" containerID="7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"
Jul 24 22:55:13.259478 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259461    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"} err="failed to get container status \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": rpc error: code = NotFound desc = could not find container \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": container with ID starting with 7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f not found: ID does not exist"
Jul 24 22:55:13.259526 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259479    2568 scope.go:117] "RemoveContainer" containerID="7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"
Jul 24 22:55:13.259526 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259479    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 22:55:13.259740 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259720    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"} err="failed to get container status \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": rpc error: code = NotFound desc = could not find container \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": container with ID starting with 7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6 not found: ID does not exist"
Jul 24 22:55:13.259787 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259744    2568 scope.go:117] "RemoveContainer" containerID="e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"
Jul 24 22:55:13.259838 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259799    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy-web"
Jul 24 22:55:13.259838 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259811    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy-web"
Jul 24 22:55:13.259838 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259820    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="init-config-reloader"
Jul 24 22:55:13.259838 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259826    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="init-config-reloader"
Jul 24 22:55:13.259838 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259835    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259843    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259854    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="thanos-sidecar"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259859    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="thanos-sidecar"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259864    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy-thanos"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259869    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy-thanos"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259878    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="prometheus"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259883    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="prometheus"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259908    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="config-reloader"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259915    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="config-reloader"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259964    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="prometheus"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259972    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="thanos-sidecar"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259978    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy-thanos"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259970    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"} err="failed to get container status \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": rpc error: code = NotFound desc = could not find container \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": container with ID starting with e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193 not found: ID does not exist"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.260000    2568 scope.go:117] "RemoveContainer" containerID="7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"
Jul 24 22:55:13.260034 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.259983    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy-web"
Jul 24 22:55:13.260634 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.260053    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="kube-rbac-proxy"
Jul 24 22:55:13.260634 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.260063    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb03511c-4e59-4ebb-9e12-2aa51c469ca2" containerName="config-reloader"
Jul 24 22:55:13.260634 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.260224    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"} err="failed to get container status \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": rpc error: code = NotFound desc = could not find container \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": container with ID starting with 7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c not found: ID does not exist"
Jul 24 22:55:13.260634 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.260243    2568 scope.go:117] "RemoveContainer" containerID="105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"
Jul 24 22:55:13.260634 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.260501    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"} err="failed to get container status \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": rpc error: code = NotFound desc = could not find container \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": container with ID starting with 105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49 not found: ID does not exist"
Jul 24 22:55:13.260634 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.260533    2568 scope.go:117] "RemoveContainer" containerID="0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"
Jul 24 22:55:13.260838 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.260751    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"} err="failed to get container status \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": rpc error: code = NotFound desc = could not find container \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": container with ID starting with 0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6 not found: ID does not exist"
Jul 24 22:55:13.260838 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.260768    2568 scope.go:117] "RemoveContainer" containerID="f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"
Jul 24 22:55:13.261011 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.260995    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"} err="failed to get container status \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": rpc error: code = NotFound desc = could not find container \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": container with ID starting with f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa not found: ID does not exist"
Jul 24 22:55:13.261051 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.261010    2568 scope.go:117] "RemoveContainer" containerID="7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"
Jul 24 22:55:13.261222 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.261206    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"} err="failed to get container status \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": rpc error: code = NotFound desc = could not find container \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": container with ID starting with 7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f not found: ID does not exist"
Jul 24 22:55:13.261285 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.261223    2568 scope.go:117] "RemoveContainer" containerID="7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"
Jul 24 22:55:13.261478 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.261454    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"} err="failed to get container status \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": rpc error: code = NotFound desc = could not find container \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": container with ID starting with 7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6 not found: ID does not exist"
Jul 24 22:55:13.261582 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.261479    2568 scope.go:117] "RemoveContainer" containerID="e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"
Jul 24 22:55:13.261730 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.261713    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"} err="failed to get container status \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": rpc error: code = NotFound desc = could not find container \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": container with ID starting with e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193 not found: ID does not exist"
Jul 24 22:55:13.261777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.261731    2568 scope.go:117] "RemoveContainer" containerID="7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"
Jul 24 22:55:13.261955 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.261935    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"} err="failed to get container status \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": rpc error: code = NotFound desc = could not find container \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": container with ID starting with 7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c not found: ID does not exist"
Jul 24 22:55:13.261955 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.261955    2568 scope.go:117] "RemoveContainer" containerID="105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"
Jul 24 22:55:13.262189 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.262163    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"} err="failed to get container status \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": rpc error: code = NotFound desc = could not find container \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": container with ID starting with 105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49 not found: ID does not exist"
Jul 24 22:55:13.262243 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.262190    2568 scope.go:117] "RemoveContainer" containerID="0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"
Jul 24 22:55:13.262450 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.262428    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"} err="failed to get container status \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": rpc error: code = NotFound desc = could not find container \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": container with ID starting with 0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6 not found: ID does not exist"
Jul 24 22:55:13.262522 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.262453    2568 scope.go:117] "RemoveContainer" containerID="f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"
Jul 24 22:55:13.262776 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.262748    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"} err="failed to get container status \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": rpc error: code = NotFound desc = could not find container \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": container with ID starting with f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa not found: ID does not exist"
Jul 24 22:55:13.262873 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.262777    2568 scope.go:117] "RemoveContainer" containerID="7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"
Jul 24 22:55:13.263067 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.263044    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"} err="failed to get container status \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": rpc error: code = NotFound desc = could not find container \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": container with ID starting with 7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f not found: ID does not exist"
Jul 24 22:55:13.263067 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.263068    2568 scope.go:117] "RemoveContainer" containerID="7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"
Jul 24 22:55:13.263319 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.263300    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"} err="failed to get container status \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": rpc error: code = NotFound desc = could not find container \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": container with ID starting with 7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6 not found: ID does not exist"
Jul 24 22:55:13.263401 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.263321    2568 scope.go:117] "RemoveContainer" containerID="e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"
Jul 24 22:55:13.263592 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.263576    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"} err="failed to get container status \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": rpc error: code = NotFound desc = could not find container \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": container with ID starting with e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193 not found: ID does not exist"
Jul 24 22:55:13.263667 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.263595    2568 scope.go:117] "RemoveContainer" containerID="7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"
Jul 24 22:55:13.263828 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.263810    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"} err="failed to get container status \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": rpc error: code = NotFound desc = could not find container \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": container with ID starting with 7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c not found: ID does not exist"
Jul 24 22:55:13.263874 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.263828    2568 scope.go:117] "RemoveContainer" containerID="105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"
Jul 24 22:55:13.263917 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.263906    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.264135 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.264113    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"} err="failed to get container status \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": rpc error: code = NotFound desc = could not find container \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": container with ID starting with 105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49 not found: ID does not exist"
Jul 24 22:55:13.264335 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.264226    2568 scope.go:117] "RemoveContainer" containerID="0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"
Jul 24 22:55:13.264601 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.264562    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"} err="failed to get container status \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": rpc error: code = NotFound desc = could not find container \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": container with ID starting with 0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6 not found: ID does not exist"
Jul 24 22:55:13.264601 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.264591    2568 scope.go:117] "RemoveContainer" containerID="f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"
Jul 24 22:55:13.264846 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.264829    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"} err="failed to get container status \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": rpc error: code = NotFound desc = could not find container \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": container with ID starting with f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa not found: ID does not exist"
Jul 24 22:55:13.264898 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.264846    2568 scope.go:117] "RemoveContainer" containerID="7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"
Jul 24 22:55:13.265080 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.265063    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"} err="failed to get container status \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": rpc error: code = NotFound desc = could not find container \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": container with ID starting with 7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f not found: ID does not exist"
Jul 24 22:55:13.265080 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.265079    2568 scope.go:117] "RemoveContainer" containerID="7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"
Jul 24 22:55:13.265295 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.265272    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6"} err="failed to get container status \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": rpc error: code = NotFound desc = could not find container \"7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6\": container with ID starting with 7e1d4fdaf1582243b86f266e2b788c47c7e0b281b4c433af6eeb05edac8ae4f6 not found: ID does not exist"
Jul 24 22:55:13.265354 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.265298    2568 scope.go:117] "RemoveContainer" containerID="e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"
Jul 24 22:55:13.265558 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.265541    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193"} err="failed to get container status \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": rpc error: code = NotFound desc = could not find container \"e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193\": container with ID starting with e73b594c9a6addcea45c5e47c958528431951072d3ab9ed0528d9f6401d4a193 not found: ID does not exist"
Jul 24 22:55:13.265600 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.265559    2568 scope.go:117] "RemoveContainer" containerID="7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"
Jul 24 22:55:13.265777 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.265760    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c"} err="failed to get container status \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": rpc error: code = NotFound desc = could not find container \"7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c\": container with ID starting with 7b8e6666e939660f982bdcc617939eac455daeb7a679907a9597d29940174a7c not found: ID does not exist"
Jul 24 22:55:13.265829 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.265778    2568 scope.go:117] "RemoveContainer" containerID="105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"
Jul 24 22:55:13.265981 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.265966    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49"} err="failed to get container status \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": rpc error: code = NotFound desc = could not find container \"105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49\": container with ID starting with 105b233ad3834aca19d7076defa6ee0c0cc7489c81bc9f3842875d46df63eb49 not found: ID does not exist"
Jul 24 22:55:13.266030 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.265981    2568 scope.go:117] "RemoveContainer" containerID="0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"
Jul 24 22:55:13.266196 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.266180    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6"} err="failed to get container status \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": rpc error: code = NotFound desc = could not find container \"0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6\": container with ID starting with 0b430bbfafb10efbb1d0951d168ad4720c92eb9c002b9670bcac856ff2cbf1d6 not found: ID does not exist"
Jul 24 22:55:13.266236 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.266196    2568 scope.go:117] "RemoveContainer" containerID="f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"
Jul 24 22:55:13.266477 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.266445    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa"} err="failed to get container status \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": rpc error: code = NotFound desc = could not find container \"f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa\": container with ID starting with f301ffa8e528fe974b4aaab3432a41f6e00e079f840be42d1ee06bb73fde93fa not found: ID does not exist"
Jul 24 22:55:13.266583 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.266477    2568 scope.go:117] "RemoveContainer" containerID="7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"
Jul 24 22:55:13.266799 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.266781    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 22:55:13.266895 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.266822    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f"} err="failed to get container status \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": rpc error: code = NotFound desc = could not find container \"7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f\": container with ID starting with 7ad25a1dc2b112aecb88416168dd4377b24a388cee315b2b196d0c8298a9032f not found: ID does not exist"
Jul 24 22:55:13.266895 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.266844    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 22:55:13.266895 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.266889    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 22:55:13.267056 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267008    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 22:55:13.267109 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267083    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-79xf6\""
Jul 24 22:55:13.267326 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267305    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 22:55:13.267398 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267333    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 22:55:13.267398 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267307    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fs4nelmbibmpl\""
Jul 24 22:55:13.267498 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267438    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 22:55:13.267569 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267514    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 22:55:13.267647 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267570    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 22:55:13.267694 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267652    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 22:55:13.267694 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267659    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 22:55:13.267830 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.267805    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 22:55:13.270001 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.269982    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 22:55:13.272865 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.272848    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 22:55:13.278906 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.278883    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 22:55:13.324554 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324520    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpxhv\" (UniqueName: \"kubernetes.io/projected/5a2c442b-2c28-4a45-bea5-825c68803d0c-kube-api-access-vpxhv\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324583    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324679    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324699    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324855 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324734    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a2c442b-2c28-4a45-bea5-825c68803d0c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324855 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324752    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324855 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324768    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324855 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324800    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324855 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324842    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-config\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324873    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324890    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324921    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324957    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.324992 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.324973    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.325222 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.325010    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.325222 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.325069    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.325222 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.325105    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.325222 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.325132    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.325222 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.325161    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a2c442b-2c28-4a45-bea5-825c68803d0c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.425880 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.425801    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.425880 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.425833    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.425880 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.425862    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.425880 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.425879    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.425896    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.425916    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.425941    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a2c442b-2c28-4a45-bea5-825c68803d0c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.425980    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpxhv\" (UniqueName: \"kubernetes.io/projected/5a2c442b-2c28-4a45-bea5-825c68803d0c-kube-api-access-vpxhv\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426023    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426048    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426072    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426127    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a2c442b-2c28-4a45-bea5-825c68803d0c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426151    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426168 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426172    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426202    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426231    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-config\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426293    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426316    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426352    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.426719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426646    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.427020 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.426727    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.427080 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.427040    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.429113 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.429083    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.429292 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.429243    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.429559 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.429436    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a2c442b-2c28-4a45-bea5-825c68803d0c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.429651 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.429613    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.429711 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.429696    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.429935 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.429914    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-config\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.429935 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.429928    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.430479 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.430272    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a2c442b-2c28-4a45-bea5-825c68803d0c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.430479 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.430428    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.430479 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.430435    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.430677 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.430504    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.430960 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.430915    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2c442b-2c28-4a45-bea5-825c68803d0c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.431990 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.431957    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.432125 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.432104    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.432471 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.432455    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.432557 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.432542    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a2c442b-2c28-4a45-bea5-825c68803d0c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.436683 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.436660    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpxhv\" (UniqueName: \"kubernetes.io/projected/5a2c442b-2c28-4a45-bea5-825c68803d0c-kube-api-access-vpxhv\") pod \"prometheus-k8s-0\" (UID: \"5a2c442b-2c28-4a45-bea5-825c68803d0c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.574325 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.574297    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:55:13.717479 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:13.717456    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 22:55:13.719797 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:55:13.719770    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2c442b_2c28_4a45_bea5_825c68803d0c.slice/crio-b3676215ecb8b4e2b5b847b4b7760f416ea76b2cf4c317f4f738b2e0626e6f27 WatchSource:0}: Error finding container b3676215ecb8b4e2b5b847b4b7760f416ea76b2cf4c317f4f738b2e0626e6f27: Status 404 returned error can't find the container with id b3676215ecb8b4e2b5b847b4b7760f416ea76b2cf4c317f4f738b2e0626e6f27
Jul 24 22:55:14.209696 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:14.209658    2568 generic.go:356] "Generic (PLEG): container finished" podID="5a2c442b-2c28-4a45-bea5-825c68803d0c" containerID="5e471136fa903841c02717622551ee76fdfcbb105fbd5d1f4d4c9cddf2145ded" exitCode=0
Jul 24 22:55:14.210063 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:14.209713    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2c442b-2c28-4a45-bea5-825c68803d0c","Type":"ContainerDied","Data":"5e471136fa903841c02717622551ee76fdfcbb105fbd5d1f4d4c9cddf2145ded"}
Jul 24 22:55:14.210063 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:14.209794    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2c442b-2c28-4a45-bea5-825c68803d0c","Type":"ContainerStarted","Data":"b3676215ecb8b4e2b5b847b4b7760f416ea76b2cf4c317f4f738b2e0626e6f27"}
Jul 24 22:55:15.218365 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:15.218331    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2c442b-2c28-4a45-bea5-825c68803d0c","Type":"ContainerStarted","Data":"33f9a172671ec18ce53a5658646e950543b2fa7df2c5fc620b8709b43f1f7de0"}
Jul 24 22:55:15.218843 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:15.218395    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2c442b-2c28-4a45-bea5-825c68803d0c","Type":"ContainerStarted","Data":"eab2fcaddf3126bc4d1647509008d2d2317bd5a4b77fc1b616d6b4936292e4d7"}
Jul 24 22:55:15.218843 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:15.218424    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2c442b-2c28-4a45-bea5-825c68803d0c","Type":"ContainerStarted","Data":"9fa05dfb052f767d69481c5ee080888a96b5038b7eaeec00a95def57a4e42d07"}
Jul 24 22:55:15.218843 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:15.218448    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2c442b-2c28-4a45-bea5-825c68803d0c","Type":"ContainerStarted","Data":"d13ff56ed26a2b6886fb49020a88382d9eb23b1208fb165171d407d2555c721f"}
Jul 24 22:55:15.218843 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:15.218470    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2c442b-2c28-4a45-bea5-825c68803d0c","Type":"ContainerStarted","Data":"947428ac71af341be1e0f4ee3340071470df7292c3fb3e8d9ed50494a4d694b1"}
Jul 24 22:55:15.218843 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:15.218493    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2c442b-2c28-4a45-bea5-825c68803d0c","Type":"ContainerStarted","Data":"f31e5eeb45d8ba34e598e668d9aec5711744fbc286a06855e4f6f04ec9c10a7a"}
Jul 24 22:55:15.247637 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:15.247580    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.229498325 podStartE2EDuration="2.247537708s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.039383ms" isStatefulPod=false podCreationTimestamp="2026-07-24 22:55:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:55:15.245407882 +0000 UTC m=+172.522441476" watchObservedRunningTime="2026-07-24 22:55:15.247537708 +0000 UTC m=+172.524571252"
Jul 24 22:55:18.574458 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:55:18.574411    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:56:13.574809 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:56:13.574771    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:56:13.590221 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:56:13.590194    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:56:14.455880 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:56:14.455856    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 22:57:03.793928 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.793891    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"]
Jul 24 22:57:03.797237 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.797216    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.800571 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.800547    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 22:57:03.800692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.800621    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 22:57:03.800692 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.800637    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7r678\""
Jul 24 22:57:03.800806 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.800795    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 22:57:03.800959 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.800943    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 22:57:03.817281 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.817260    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"]
Jul 24 22:57:03.885982 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.885942    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26157995-be18-49f3-b427-2428485abb49-tmp\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.885982 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.885981    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/26157995-be18-49f3-b427-2428485abb49-manifests\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.886197 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.886037    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26157995-be18-49f3-b427-2428485abb49-webhook-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.886197 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.886082    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/26157995-be18-49f3-b427-2428485abb49-apiservice-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.886197 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.886105    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjgkv\" (UniqueName: \"kubernetes.io/projected/26157995-be18-49f3-b427-2428485abb49-kube-api-access-cjgkv\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.987287 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.987230    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26157995-be18-49f3-b427-2428485abb49-webhook-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.987430 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.987317    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/26157995-be18-49f3-b427-2428485abb49-apiservice-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.987430 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.987344    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjgkv\" (UniqueName: \"kubernetes.io/projected/26157995-be18-49f3-b427-2428485abb49-kube-api-access-cjgkv\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.987430 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.987400    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26157995-be18-49f3-b427-2428485abb49-tmp\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.987654 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.987428    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/26157995-be18-49f3-b427-2428485abb49-manifests\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.987828 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.987808    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/26157995-be18-49f3-b427-2428485abb49-manifests\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.989551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.989528    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26157995-be18-49f3-b427-2428485abb49-tmp\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.990189 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.990174    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/26157995-be18-49f3-b427-2428485abb49-apiservice-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.990309 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.990290    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26157995-be18-49f3-b427-2428485abb49-webhook-cert\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:03.995386 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:03.995365    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjgkv\" (UniqueName: \"kubernetes.io/projected/26157995-be18-49f3-b427-2428485abb49-kube-api-access-cjgkv\") pod \"opendatahub-operator-controller-manager-777c88b5cf-q5lhr\" (UID: \"26157995-be18-49f3-b427-2428485abb49\") " pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:04.109005 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:04.108915    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:04.240592 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:04.240563    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"]
Jul 24 22:57:04.244153 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:57:04.244125    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26157995_be18_49f3_b427_2428485abb49.slice/crio-dcf1ad7984b81f2227b3cff4100c17a2d68459aa179b144a5085528c4bbafd8b WatchSource:0}: Error finding container dcf1ad7984b81f2227b3cff4100c17a2d68459aa179b144a5085528c4bbafd8b: Status 404 returned error can't find the container with id dcf1ad7984b81f2227b3cff4100c17a2d68459aa179b144a5085528c4bbafd8b
Jul 24 22:57:04.630001 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:04.629963    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr" event={"ID":"26157995-be18-49f3-b427-2428485abb49","Type":"ContainerStarted","Data":"dcf1ad7984b81f2227b3cff4100c17a2d68459aa179b144a5085528c4bbafd8b"}
Jul 24 22:57:07.646847 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:07.646809    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr" event={"ID":"26157995-be18-49f3-b427-2428485abb49","Type":"ContainerStarted","Data":"a105a468f4903fd8b0f4d76df3e9f27b93b67d70bc23985a0ea1cd06be4f1d9e"}
Jul 24 22:57:08.652281 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:08.652187    2568 generic.go:356] "Generic (PLEG): container finished" podID="26157995-be18-49f3-b427-2428485abb49" containerID="a105a468f4903fd8b0f4d76df3e9f27b93b67d70bc23985a0ea1cd06be4f1d9e" exitCode=0
Jul 24 22:57:08.652281 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:08.652272    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr" event={"ID":"26157995-be18-49f3-b427-2428485abb49","Type":"ContainerDied","Data":"a105a468f4903fd8b0f4d76df3e9f27b93b67d70bc23985a0ea1cd06be4f1d9e"}
Jul 24 22:57:09.658323 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:09.658288    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr" event={"ID":"26157995-be18-49f3-b427-2428485abb49","Type":"ContainerStarted","Data":"f06f5a4e05a257eac996814dd9ba9aea148bdb6351a5a5848c84ea0d049757d8"}
Jul 24 22:57:09.658703 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:09.658426    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:09.683479 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:09.683421    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr" podStartSLOduration=2.277894908 podStartE2EDuration="6.683401594s" totalImagesPullingTime="3.365224494s" totalInitContainerRuntime="1.040282192s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:09.680930665 +0000 UTC m=+286.957964210" watchObservedRunningTime="2026-07-24 22:57:09.683401594 +0000 UTC m=+286.960435141"
Jul 24 22:57:20.664874 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:20.664840    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-777c88b5cf-q5lhr"
Jul 24 22:57:21.845757 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.845722    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"]
Jul 24 22:57:21.852568 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.852545    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:21.855287 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.855271    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 22:57:21.856054 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.856040    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 22:57:21.856144 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.856122    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 22:57:21.856203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.856172    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 22:57:21.856285 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.856196    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rfbnv\""
Jul 24 22:57:21.856285 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.856240    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 22:57:21.861523 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.861500    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"]
Jul 24 22:57:21.937881 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.937850    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/da543487-63fd-4dc0-8303-bedd8496c164-metrics-cert\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:21.937881 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.937880    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/da543487-63fd-4dc0-8303-bedd8496c164-manager-config\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:21.938097 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.937912    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksm5z\" (UniqueName: \"kubernetes.io/projected/da543487-63fd-4dc0-8303-bedd8496c164-kube-api-access-ksm5z\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:21.938097 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:21.937951    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da543487-63fd-4dc0-8303-bedd8496c164-cert\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:22.039350 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.039318    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da543487-63fd-4dc0-8303-bedd8496c164-cert\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:22.039450 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.039389    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/da543487-63fd-4dc0-8303-bedd8496c164-metrics-cert\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:22.039450 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.039410    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/da543487-63fd-4dc0-8303-bedd8496c164-manager-config\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:22.039450 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.039442    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksm5z\" (UniqueName: \"kubernetes.io/projected/da543487-63fd-4dc0-8303-bedd8496c164-kube-api-access-ksm5z\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:22.040071 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.040050    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/da543487-63fd-4dc0-8303-bedd8496c164-manager-config\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:22.041859 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.041835    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/da543487-63fd-4dc0-8303-bedd8496c164-metrics-cert\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:22.041859 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.041857    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da543487-63fd-4dc0-8303-bedd8496c164-cert\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:22.048316 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.048295    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksm5z\" (UniqueName: \"kubernetes.io/projected/da543487-63fd-4dc0-8303-bedd8496c164-kube-api-access-ksm5z\") pod \"lws-controller-manager-6bccc7ffcc-s4t5m\" (UID: \"da543487-63fd-4dc0-8303-bedd8496c164\") " pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:22.162549 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.162458    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:22.297860 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.297836    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"]
Jul 24 22:57:22.300260 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:57:22.300223    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda543487_63fd_4dc0_8303_bedd8496c164.slice/crio-a0c589a1f5e96c191919e8a223db220c89111859b3a982e62a47e7e8ada9341c WatchSource:0}: Error finding container a0c589a1f5e96c191919e8a223db220c89111859b3a982e62a47e7e8ada9341c: Status 404 returned error can't find the container with id a0c589a1f5e96c191919e8a223db220c89111859b3a982e62a47e7e8ada9341c
Jul 24 22:57:22.715870 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:22.715832    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m" event={"ID":"da543487-63fd-4dc0-8303-bedd8496c164","Type":"ContainerStarted","Data":"a0c589a1f5e96c191919e8a223db220c89111859b3a982e62a47e7e8ada9341c"}
Jul 24 22:57:23.106621 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.106525    2568 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 22:57:23.216820 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.216790    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"]
Jul 24 22:57:23.220659 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.220640    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.224103 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.223966    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 22:57:23.224103 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.223970    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 22:57:23.225170 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.225134    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hnqs9\""
Jul 24 22:57:23.225279 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.225191    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 22:57:23.225405 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.225389    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 22:57:23.239353 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.239327    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"]
Jul 24 22:57:23.248523 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.248489    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3361a14a-c83a-4060-8107-33c16983c567-tmp\") pod \"kube-auth-proxy-745f8948c6-6sd86\" (UID: \"3361a14a-c83a-4060-8107-33c16983c567\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.248673 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.248570    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6rp8\" (UniqueName: \"kubernetes.io/projected/3361a14a-c83a-4060-8107-33c16983c567-kube-api-access-f6rp8\") pod \"kube-auth-proxy-745f8948c6-6sd86\" (UID: \"3361a14a-c83a-4060-8107-33c16983c567\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.248673 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.248623    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3361a14a-c83a-4060-8107-33c16983c567-tls-certs\") pod \"kube-auth-proxy-745f8948c6-6sd86\" (UID: \"3361a14a-c83a-4060-8107-33c16983c567\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.349551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.349491    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3361a14a-c83a-4060-8107-33c16983c567-tls-certs\") pod \"kube-auth-proxy-745f8948c6-6sd86\" (UID: \"3361a14a-c83a-4060-8107-33c16983c567\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.349766 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.349618    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3361a14a-c83a-4060-8107-33c16983c567-tmp\") pod \"kube-auth-proxy-745f8948c6-6sd86\" (UID: \"3361a14a-c83a-4060-8107-33c16983c567\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.349766 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.349683    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6rp8\" (UniqueName: \"kubernetes.io/projected/3361a14a-c83a-4060-8107-33c16983c567-kube-api-access-f6rp8\") pod \"kube-auth-proxy-745f8948c6-6sd86\" (UID: \"3361a14a-c83a-4060-8107-33c16983c567\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.352509 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.352484    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3361a14a-c83a-4060-8107-33c16983c567-tmp\") pod \"kube-auth-proxy-745f8948c6-6sd86\" (UID: \"3361a14a-c83a-4060-8107-33c16983c567\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.352801 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.352761    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3361a14a-c83a-4060-8107-33c16983c567-tls-certs\") pod \"kube-auth-proxy-745f8948c6-6sd86\" (UID: \"3361a14a-c83a-4060-8107-33c16983c567\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.360571 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.360454    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6rp8\" (UniqueName: \"kubernetes.io/projected/3361a14a-c83a-4060-8107-33c16983c567-kube-api-access-f6rp8\") pod \"kube-auth-proxy-745f8948c6-6sd86\" (UID: \"3361a14a-c83a-4060-8107-33c16983c567\") " pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.534818 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.534778    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"
Jul 24 22:57:23.688805 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.688630    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-745f8948c6-6sd86"]
Jul 24 22:57:23.691862 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:57:23.691814    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3361a14a_c83a_4060_8107_33c16983c567.slice/crio-5903eae73f18a4606ecf1e18b7b1e7bcd5a5e92053bcff3c1b88709922dc093e WatchSource:0}: Error finding container 5903eae73f18a4606ecf1e18b7b1e7bcd5a5e92053bcff3c1b88709922dc093e: Status 404 returned error can't find the container with id 5903eae73f18a4606ecf1e18b7b1e7bcd5a5e92053bcff3c1b88709922dc093e
Jul 24 22:57:23.721342 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:23.721300    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86" event={"ID":"3361a14a-c83a-4060-8107-33c16983c567","Type":"ContainerStarted","Data":"5903eae73f18a4606ecf1e18b7b1e7bcd5a5e92053bcff3c1b88709922dc093e"}
Jul 24 22:57:25.734139 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:25.734099    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m" event={"ID":"da543487-63fd-4dc0-8303-bedd8496c164","Type":"ContainerStarted","Data":"a9a11708675ad1322385eb4775bdc298361bb6437391da8a222e8defaa570a44"}
Jul 24 22:57:25.734571 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:25.734195    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:57:25.765024 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:25.764970    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m" podStartSLOduration=2.21208525 podStartE2EDuration="4.764951402s" totalImagesPullingTime="2.552866152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:25.764322725 +0000 UTC m=+303.041356270" watchObservedRunningTime="2026-07-24 22:57:25.764951402 +0000 UTC m=+303.041984987"
Jul 24 22:57:26.739521 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:26.739476    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86" event={"ID":"3361a14a-c83a-4060-8107-33c16983c567","Type":"ContainerStarted","Data":"14ab4f74e0c8bb91c423488ddc17b1a73d9fd2b013cef5e71ef6d84837f39584"}
Jul 24 22:57:26.757440 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:26.757388    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-745f8948c6-6sd86" podStartSLOduration=1.103918518 podStartE2EDuration="3.757373283s" totalImagesPullingTime="2.653454765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:57:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:57:26.756422832 +0000 UTC m=+304.033456375" watchObservedRunningTime="2026-07-24 22:57:26.757373283 +0000 UTC m=+304.034406827"
Jul 24 22:57:36.741766 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:57:36.741734    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6bccc7ffcc-s4t5m"
Jul 24 22:59:07.544943 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.544906    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"]
Jul 24 22:59:07.548070 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.548052    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:07.550583 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.550561    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 22:59:07.551675 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.551649    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 24 22:59:07.551783 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.551689    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 22:59:07.551783 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.551689    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6p7bk\""
Jul 24 22:59:07.551783 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.551721    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 24 22:59:07.558381 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.558361    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"]
Jul 24 22:59:07.661739 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.661708    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c931f7c-6c3c-4ff4-b7ab-278ef80f0798-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-g2tfk\" (UID: \"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:07.661904 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.661751    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6jlt\" (UniqueName: \"kubernetes.io/projected/4c931f7c-6c3c-4ff4-b7ab-278ef80f0798-kube-api-access-q6jlt\") pod \"kuadrant-console-plugin-66b5774985-g2tfk\" (UID: \"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:07.661904 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.661847    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c931f7c-6c3c-4ff4-b7ab-278ef80f0798-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-g2tfk\" (UID: \"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:07.762985 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.762950    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c931f7c-6c3c-4ff4-b7ab-278ef80f0798-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-g2tfk\" (UID: \"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:07.763166 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.763000    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c931f7c-6c3c-4ff4-b7ab-278ef80f0798-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-g2tfk\" (UID: \"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:07.763166 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.763032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6jlt\" (UniqueName: \"kubernetes.io/projected/4c931f7c-6c3c-4ff4-b7ab-278ef80f0798-kube-api-access-q6jlt\") pod \"kuadrant-console-plugin-66b5774985-g2tfk\" (UID: \"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:07.763759 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.763738    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c931f7c-6c3c-4ff4-b7ab-278ef80f0798-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-g2tfk\" (UID: \"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:07.765587 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.765567    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c931f7c-6c3c-4ff4-b7ab-278ef80f0798-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-g2tfk\" (UID: \"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:07.777455 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.777424    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6jlt\" (UniqueName: \"kubernetes.io/projected/4c931f7c-6c3c-4ff4-b7ab-278ef80f0798-kube-api-access-q6jlt\") pod \"kuadrant-console-plugin-66b5774985-g2tfk\" (UID: \"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:07.871567 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:07.871477    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"
Jul 24 22:59:08.000638 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:08.000615    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk"]
Jul 24 22:59:08.003228 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:59:08.003194    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c931f7c_6c3c_4ff4_b7ab_278ef80f0798.slice/crio-db8da78edc8581b129e136fbb5c9737f10dc2b8bbf58e4d69264970094f4c890 WatchSource:0}: Error finding container db8da78edc8581b129e136fbb5c9737f10dc2b8bbf58e4d69264970094f4c890: Status 404 returned error can't find the container with id db8da78edc8581b129e136fbb5c9737f10dc2b8bbf58e4d69264970094f4c890
Jul 24 22:59:08.004593 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:08.004572    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 22:59:08.173912 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:08.173831    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk" event={"ID":"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798","Type":"ContainerStarted","Data":"db8da78edc8581b129e136fbb5c9737f10dc2b8bbf58e4d69264970094f4c890"}
Jul 24 22:59:17.065276 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.065229    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"]
Jul 24 22:59:17.087080 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.087032    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"]
Jul 24 22:59:17.087284 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.087175    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:17.090724 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.090701    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-grhwh\""
Jul 24 22:59:17.154309 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.154211    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b4ll\" (UniqueName: \"kubernetes.io/projected/19b8c2c5-99af-47ff-9986-80375b39e7f8-kube-api-access-8b4ll\") pod \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" (UID: \"19b8c2c5-99af-47ff-9986-80375b39e7f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:17.154503 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.154322    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19b8c2c5-99af-47ff-9986-80375b39e7f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" (UID: \"19b8c2c5-99af-47ff-9986-80375b39e7f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:17.255187 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.255132    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b4ll\" (UniqueName: \"kubernetes.io/projected/19b8c2c5-99af-47ff-9986-80375b39e7f8-kube-api-access-8b4ll\") pod \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" (UID: \"19b8c2c5-99af-47ff-9986-80375b39e7f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:17.255379 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.255302    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19b8c2c5-99af-47ff-9986-80375b39e7f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" (UID: \"19b8c2c5-99af-47ff-9986-80375b39e7f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:17.255749 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.255711    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19b8c2c5-99af-47ff-9986-80375b39e7f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" (UID: \"19b8c2c5-99af-47ff-9986-80375b39e7f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:17.268990 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.268965    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b4ll\" (UniqueName: \"kubernetes.io/projected/19b8c2c5-99af-47ff-9986-80375b39e7f8-kube-api-access-8b4ll\") pod \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" (UID: \"19b8c2c5-99af-47ff-9986-80375b39e7f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:17.401218 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.401132    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:17.495178 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.495139    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"]
Jul 24 22:59:17.499260 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.499128    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"]
Jul 24 22:59:17.521562 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.521532    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"]
Jul 24 22:59:17.525334 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.525310    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:17.544733 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.544699    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"]
Jul 24 22:59:17.659068 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.658987    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0777d40e-5aa6-4a29-9913-6017fc9735df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hrs6g\" (UID: \"0777d40e-5aa6-4a29-9913-6017fc9735df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:17.659226 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.659078    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6tcd\" (UniqueName: \"kubernetes.io/projected/0777d40e-5aa6-4a29-9913-6017fc9735df-kube-api-access-v6tcd\") pod \"kuadrant-operator-controller-manager-55467d9f7-hrs6g\" (UID: \"0777d40e-5aa6-4a29-9913-6017fc9735df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:17.759637 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.759608    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6tcd\" (UniqueName: \"kubernetes.io/projected/0777d40e-5aa6-4a29-9913-6017fc9735df-kube-api-access-v6tcd\") pod \"kuadrant-operator-controller-manager-55467d9f7-hrs6g\" (UID: \"0777d40e-5aa6-4a29-9913-6017fc9735df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:17.759840 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.759680    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0777d40e-5aa6-4a29-9913-6017fc9735df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hrs6g\" (UID: \"0777d40e-5aa6-4a29-9913-6017fc9735df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:17.760083 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.760062    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0777d40e-5aa6-4a29-9913-6017fc9735df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hrs6g\" (UID: \"0777d40e-5aa6-4a29-9913-6017fc9735df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:17.772863 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.772807    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6tcd\" (UniqueName: \"kubernetes.io/projected/0777d40e-5aa6-4a29-9913-6017fc9735df-kube-api-access-v6tcd\") pod \"kuadrant-operator-controller-manager-55467d9f7-hrs6g\" (UID: \"0777d40e-5aa6-4a29-9913-6017fc9735df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:17.844660 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:17.844623    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:28.684892 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:28.684862    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"]
Jul 24 22:59:28.686839 ip-10-0-132-153 kubenswrapper[2568]: W0724 22:59:28.686816    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0777d40e_5aa6_4a29_9913_6017fc9735df.slice/crio-b7304fffea18b6af1e02d38dbc2c0d75f2ba7bcce832f17c55cd97fa182ccc73 WatchSource:0}: Error finding container b7304fffea18b6af1e02d38dbc2c0d75f2ba7bcce832f17c55cd97fa182ccc73: Status 404 returned error can't find the container with id b7304fffea18b6af1e02d38dbc2c0d75f2ba7bcce832f17c55cd97fa182ccc73
Jul 24 22:59:29.289010 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:29.288969    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk" event={"ID":"4c931f7c-6c3c-4ff4-b7ab-278ef80f0798","Type":"ContainerStarted","Data":"08923401538a4796335fc000f2d1794a7370b29823277177c0f0cee2aadba2db"}
Jul 24 22:59:29.290313 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:29.290283    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g" event={"ID":"0777d40e-5aa6-4a29-9913-6017fc9735df","Type":"ContainerStarted","Data":"b7304fffea18b6af1e02d38dbc2c0d75f2ba7bcce832f17c55cd97fa182ccc73"}
Jul 24 22:59:29.305551 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:29.305509    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g2tfk" podStartSLOduration=1.68235815 podStartE2EDuration="22.305467265s" totalImagesPullingTime="20.623109115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:59:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:59:29.304830657 +0000 UTC m=+426.581864203" watchObservedRunningTime="2026-07-24 22:59:29.305467265 +0000 UTC m=+426.582500809"
Jul 24 22:59:31.089424 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:59:31.089340    2568 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-5b8hf_kuadrant-system_19b8c2c5-99af-47ff-9986-80375b39e7f8_0(ea36bddd2bd53e3a6a3b5faaa6d76ac4fd41fd7d3b6a083c557f55e76a0f5bb9): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-5b8hf to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 24 22:59:31.089755 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:59:31.089449    2568 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-5b8hf_kuadrant-system_19b8c2c5-99af-47ff-9986-80375b39e7f8_0(ea36bddd2bd53e3a6a3b5faaa6d76ac4fd41fd7d3b6a083c557f55e76a0f5bb9): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-5b8hf to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:31.301367 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:31.301331    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:31.303829 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:59:31.303789    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" is forbidden: User \"system:node:ip-10-0-132-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-153.ec2.internal' and this object" podUID="19b8c2c5-99af-47ff-9986-80375b39e7f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:31.340773 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:31.340694    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:31.343208 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:59:31.343160    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" is forbidden: User \"system:node:ip-10-0-132-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-153.ec2.internal' and this object" podUID="19b8c2c5-99af-47ff-9986-80375b39e7f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:31.496216 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:31.496183    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8b4ll\" (UniqueName: \"kubernetes.io/projected/19b8c2c5-99af-47ff-9986-80375b39e7f8-kube-api-access-8b4ll\") pod \"19b8c2c5-99af-47ff-9986-80375b39e7f8\" (UID: \"19b8c2c5-99af-47ff-9986-80375b39e7f8\") "
Jul 24 22:59:31.496410 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:31.496241    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19b8c2c5-99af-47ff-9986-80375b39e7f8-extensions-socket-volume\") pod \"19b8c2c5-99af-47ff-9986-80375b39e7f8\" (UID: \"19b8c2c5-99af-47ff-9986-80375b39e7f8\") "
Jul 24 22:59:31.496586 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:31.496520    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b8c2c5-99af-47ff-9986-80375b39e7f8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "19b8c2c5-99af-47ff-9986-80375b39e7f8" (UID: "19b8c2c5-99af-47ff-9986-80375b39e7f8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:59:31.496705 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:31.496636    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19b8c2c5-99af-47ff-9986-80375b39e7f8-extensions-socket-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:59:31.498712 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:31.498686    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19b8c2c5-99af-47ff-9986-80375b39e7f8-kube-api-access-8b4ll" (OuterVolumeSpecName: "kube-api-access-8b4ll") pod "19b8c2c5-99af-47ff-9986-80375b39e7f8" (UID: "19b8c2c5-99af-47ff-9986-80375b39e7f8"). InnerVolumeSpecName "kube-api-access-8b4ll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:59:31.597867 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:31.597788    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b4ll\" (UniqueName: \"kubernetes.io/projected/19b8c2c5-99af-47ff-9986-80375b39e7f8-kube-api-access-8b4ll\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:59:32.305772 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:32.305742    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:32.308190 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:59:32.308149    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" is forbidden: User \"system:node:ip-10-0-132-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-153.ec2.internal' and this object" podUID="19b8c2c5-99af-47ff-9986-80375b39e7f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:32.315054 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:59:32.315018    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" is forbidden: User \"system:node:ip-10-0-132-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-153.ec2.internal' and this object" podUID="19b8c2c5-99af-47ff-9986-80375b39e7f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:33.256887 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:59:33.256805    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5b8hf\" is forbidden: User \"system:node:ip-10-0-132-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-153.ec2.internal' and this object" podUID="19b8c2c5-99af-47ff-9986-80375b39e7f8" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5b8hf"
Jul 24 22:59:33.257671 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:33.257645    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19b8c2c5-99af-47ff-9986-80375b39e7f8" path="/var/lib/kubelet/pods/19b8c2c5-99af-47ff-9986-80375b39e7f8/volumes"
Jul 24 22:59:33.311790 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:33.311750    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g" event={"ID":"0777d40e-5aa6-4a29-9913-6017fc9735df","Type":"ContainerStarted","Data":"b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371"}
Jul 24 22:59:33.312203 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:33.311888    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:33.330719 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:33.330650    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g" podStartSLOduration=12.026715648 podStartE2EDuration="16.330635692s" totalImagesPullingTime="4.303920044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:59:33.328936853 +0000 UTC m=+430.605970418" watchObservedRunningTime="2026-07-24 22:59:33.330635692 +0000 UTC m=+430.607669235"
Jul 24 22:59:44.318613 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:44.318582    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:57.046241 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.046208    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"]
Jul 24 22:59:57.046712 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.046449    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g" podUID="0777d40e-5aa6-4a29-9913-6017fc9735df" containerName="manager" containerID="cri-o://b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371" gracePeriod=10
Jul 24 22:59:57.295680 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.295652    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:57.416909 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.416825    2568 generic.go:356] "Generic (PLEG): container finished" podID="0777d40e-5aa6-4a29-9913-6017fc9735df" containerID="b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371" exitCode=0
Jul 24 22:59:57.416909 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.416871    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"
Jul 24 22:59:57.417085 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.416907    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g" event={"ID":"0777d40e-5aa6-4a29-9913-6017fc9735df","Type":"ContainerDied","Data":"b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371"}
Jul 24 22:59:57.417085 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.416964    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g" event={"ID":"0777d40e-5aa6-4a29-9913-6017fc9735df","Type":"ContainerDied","Data":"b7304fffea18b6af1e02d38dbc2c0d75f2ba7bcce832f17c55cd97fa182ccc73"}
Jul 24 22:59:57.417085 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.416988    2568 scope.go:117] "RemoveContainer" containerID="b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371"
Jul 24 22:59:57.425378 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.425359    2568 scope.go:117] "RemoveContainer" containerID="b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371"
Jul 24 22:59:57.425520 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.425498    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0777d40e-5aa6-4a29-9913-6017fc9735df-extensions-socket-volume\") pod \"0777d40e-5aa6-4a29-9913-6017fc9735df\" (UID: \"0777d40e-5aa6-4a29-9913-6017fc9735df\") "
Jul 24 22:59:57.425636 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.425529    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6tcd\" (UniqueName: \"kubernetes.io/projected/0777d40e-5aa6-4a29-9913-6017fc9735df-kube-api-access-v6tcd\") pod \"0777d40e-5aa6-4a29-9913-6017fc9735df\" (UID: \"0777d40e-5aa6-4a29-9913-6017fc9735df\") "
Jul 24 22:59:57.425718 ip-10-0-132-153 kubenswrapper[2568]: E0724 22:59:57.425694    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371\": container with ID starting with b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371 not found: ID does not exist" containerID="b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371"
Jul 24 22:59:57.425775 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.425731    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371"} err="failed to get container status \"b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371\": rpc error: code = NotFound desc = could not find container \"b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371\": container with ID starting with b47ad3e0ed0a421bd33abd1bba4b06b43eaa43c429e47d181aaba2ee390bf371 not found: ID does not exist"
Jul 24 22:59:57.425963 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.425943    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0777d40e-5aa6-4a29-9913-6017fc9735df-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0777d40e-5aa6-4a29-9913-6017fc9735df" (UID: "0777d40e-5aa6-4a29-9913-6017fc9735df"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 22:59:57.427595 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.427567    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0777d40e-5aa6-4a29-9913-6017fc9735df-kube-api-access-v6tcd" (OuterVolumeSpecName: "kube-api-access-v6tcd") pod "0777d40e-5aa6-4a29-9913-6017fc9735df" (UID: "0777d40e-5aa6-4a29-9913-6017fc9735df"). InnerVolumeSpecName "kube-api-access-v6tcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 22:59:57.527064 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.527031    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0777d40e-5aa6-4a29-9913-6017fc9735df-extensions-socket-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:59:57.527064 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.527058    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6tcd\" (UniqueName: \"kubernetes.io/projected/0777d40e-5aa6-4a29-9913-6017fc9735df-kube-api-access-v6tcd\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 22:59:57.739438 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.739405    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"]
Jul 24 22:59:57.745923 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:57.745894    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hrs6g"]
Jul 24 22:59:59.257679 ip-10-0-132-153 kubenswrapper[2568]: I0724 22:59:59.257643    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0777d40e-5aa6-4a29-9913-6017fc9735df" path="/var/lib/kubelet/pods/0777d40e-5aa6-4a29-9913-6017fc9735df/volumes"
Jul 24 23:00:03.169905 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.169869    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6g98g"]
Jul 24 23:00:03.170316 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.170196    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0777d40e-5aa6-4a29-9913-6017fc9735df" containerName="manager"
Jul 24 23:00:03.170316 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.170208    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0777d40e-5aa6-4a29-9913-6017fc9735df" containerName="manager"
Jul 24 23:00:03.170316 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.170281    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0777d40e-5aa6-4a29-9913-6017fc9735df" containerName="manager"
Jul 24 23:00:03.196827 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.196795    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6g98g"]
Jul 24 23:00:03.196982 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.196896    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:03.201019 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.200999    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 23:00:03.280612 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.280581    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b9ce846d-c40d-43d9-8965-e85732ee6968-config-file\") pod \"limitador-limitador-64c94fd64f-6g98g\" (UID: \"b9ce846d-c40d-43d9-8965-e85732ee6968\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:03.280612 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.280612    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffkwh\" (UniqueName: \"kubernetes.io/projected/b9ce846d-c40d-43d9-8965-e85732ee6968-kube-api-access-ffkwh\") pod \"limitador-limitador-64c94fd64f-6g98g\" (UID: \"b9ce846d-c40d-43d9-8965-e85732ee6968\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:03.295836 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.295805    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6g98g"]
Jul 24 23:00:03.382026 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.381996    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b9ce846d-c40d-43d9-8965-e85732ee6968-config-file\") pod \"limitador-limitador-64c94fd64f-6g98g\" (UID: \"b9ce846d-c40d-43d9-8965-e85732ee6968\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:03.382026 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.382028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffkwh\" (UniqueName: \"kubernetes.io/projected/b9ce846d-c40d-43d9-8965-e85732ee6968-kube-api-access-ffkwh\") pod \"limitador-limitador-64c94fd64f-6g98g\" (UID: \"b9ce846d-c40d-43d9-8965-e85732ee6968\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:03.382628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.382604    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b9ce846d-c40d-43d9-8965-e85732ee6968-config-file\") pod \"limitador-limitador-64c94fd64f-6g98g\" (UID: \"b9ce846d-c40d-43d9-8965-e85732ee6968\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:03.392500 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.392476    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffkwh\" (UniqueName: \"kubernetes.io/projected/b9ce846d-c40d-43d9-8965-e85732ee6968-kube-api-access-ffkwh\") pod \"limitador-limitador-64c94fd64f-6g98g\" (UID: \"b9ce846d-c40d-43d9-8965-e85732ee6968\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:03.506154 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.506130    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:03.634718 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:03.634686    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6g98g"]
Jul 24 23:00:03.636522 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:00:03.636495    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9ce846d_c40d_43d9_8965_e85732ee6968.slice/crio-e693682e097c34dd25c3ee759ee43de5c0da32606c15e3d570b7007be3747412 WatchSource:0}: Error finding container e693682e097c34dd25c3ee759ee43de5c0da32606c15e3d570b7007be3747412: Status 404 returned error can't find the container with id e693682e097c34dd25c3ee759ee43de5c0da32606c15e3d570b7007be3747412
Jul 24 23:00:04.216113 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:04.216077    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-rtmbg"]
Jul 24 23:00:04.220436 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:04.220419    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rtmbg"
Jul 24 23:00:04.222855 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:04.222836    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lskhv\""
Jul 24 23:00:04.228963 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:04.228942    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rtmbg"]
Jul 24 23:00:04.289979 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:04.289942    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2lf4\" (UniqueName: \"kubernetes.io/projected/33c79b50-87e9-4232-a31a-06cfeb02d53b-kube-api-access-d2lf4\") pod \"authorino-75d556bd77-rtmbg\" (UID: \"33c79b50-87e9-4232-a31a-06cfeb02d53b\") " pod="kuadrant-system/authorino-75d556bd77-rtmbg"
Jul 24 23:00:04.391749 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:04.391704    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2lf4\" (UniqueName: \"kubernetes.io/projected/33c79b50-87e9-4232-a31a-06cfeb02d53b-kube-api-access-d2lf4\") pod \"authorino-75d556bd77-rtmbg\" (UID: \"33c79b50-87e9-4232-a31a-06cfeb02d53b\") " pod="kuadrant-system/authorino-75d556bd77-rtmbg"
Jul 24 23:00:04.407071 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:04.407037    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2lf4\" (UniqueName: \"kubernetes.io/projected/33c79b50-87e9-4232-a31a-06cfeb02d53b-kube-api-access-d2lf4\") pod \"authorino-75d556bd77-rtmbg\" (UID: \"33c79b50-87e9-4232-a31a-06cfeb02d53b\") " pod="kuadrant-system/authorino-75d556bd77-rtmbg"
Jul 24 23:00:04.455672 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:04.455622    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g" event={"ID":"b9ce846d-c40d-43d9-8965-e85732ee6968","Type":"ContainerStarted","Data":"e693682e097c34dd25c3ee759ee43de5c0da32606c15e3d570b7007be3747412"}
Jul 24 23:00:04.530127 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:04.530048    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rtmbg"
Jul 24 23:00:04.685524 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:04.685489    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rtmbg"]
Jul 24 23:00:04.687911 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:00:04.687876    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33c79b50_87e9_4232_a31a_06cfeb02d53b.slice/crio-ff0887a695bd8d97b0c6f37814d41f41fcb1a28938e9695ac174c1b5719710c1 WatchSource:0}: Error finding container ff0887a695bd8d97b0c6f37814d41f41fcb1a28938e9695ac174c1b5719710c1: Status 404 returned error can't find the container with id ff0887a695bd8d97b0c6f37814d41f41fcb1a28938e9695ac174c1b5719710c1
Jul 24 23:00:05.461807 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:05.461771    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rtmbg" event={"ID":"33c79b50-87e9-4232-a31a-06cfeb02d53b","Type":"ContainerStarted","Data":"ff0887a695bd8d97b0c6f37814d41f41fcb1a28938e9695ac174c1b5719710c1"}
Jul 24 23:00:07.473964 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:07.473929    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g" event={"ID":"b9ce846d-c40d-43d9-8965-e85732ee6968","Type":"ContainerStarted","Data":"23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef"}
Jul 24 23:00:07.474401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:07.474011    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:07.491410 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:07.491351    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g" podStartSLOduration=1.704555559 podStartE2EDuration="4.491328398s" totalImagesPullingTime="2.786772839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:00:07.489804117 +0000 UTC m=+464.766837662" watchObservedRunningTime="2026-07-24 23:00:07.491328398 +0000 UTC m=+464.768361943"
Jul 24 23:00:09.484648 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:09.484620    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rtmbg" event={"ID":"33c79b50-87e9-4232-a31a-06cfeb02d53b","Type":"ContainerStarted","Data":"fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99"}
Jul 24 23:00:09.500319 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:09.500274    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-rtmbg" podStartSLOduration=1.3989757329999999 podStartE2EDuration="5.500244338s" totalImagesPullingTime="4.101268605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:00:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:00:09.498968649 +0000 UTC m=+466.776002195" watchObservedRunningTime="2026-07-24 23:00:09.500244338 +0000 UTC m=+466.777277881"
Jul 24 23:00:18.480932 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:18.480899    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:20.173956 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:20.173924    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6g98g"]
Jul 24 23:00:20.174373 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:20.174105    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g" podUID="b9ce846d-c40d-43d9-8965-e85732ee6968" containerName="limitador" containerID="cri-o://23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef" gracePeriod=30
Jul 24 23:00:20.719560 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:20.719537    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:20.852554 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:20.852482    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffkwh\" (UniqueName: \"kubernetes.io/projected/b9ce846d-c40d-43d9-8965-e85732ee6968-kube-api-access-ffkwh\") pod \"b9ce846d-c40d-43d9-8965-e85732ee6968\" (UID: \"b9ce846d-c40d-43d9-8965-e85732ee6968\") "
Jul 24 23:00:20.852554 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:20.852521    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b9ce846d-c40d-43d9-8965-e85732ee6968-config-file\") pod \"b9ce846d-c40d-43d9-8965-e85732ee6968\" (UID: \"b9ce846d-c40d-43d9-8965-e85732ee6968\") "
Jul 24 23:00:20.852925 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:20.852898    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9ce846d-c40d-43d9-8965-e85732ee6968-config-file" (OuterVolumeSpecName: "config-file") pod "b9ce846d-c40d-43d9-8965-e85732ee6968" (UID: "b9ce846d-c40d-43d9-8965-e85732ee6968"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:00:20.854597 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:20.854576    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9ce846d-c40d-43d9-8965-e85732ee6968-kube-api-access-ffkwh" (OuterVolumeSpecName: "kube-api-access-ffkwh") pod "b9ce846d-c40d-43d9-8965-e85732ee6968" (UID: "b9ce846d-c40d-43d9-8965-e85732ee6968"). InnerVolumeSpecName "kube-api-access-ffkwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:00:20.953649 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:20.953621    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffkwh\" (UniqueName: \"kubernetes.io/projected/b9ce846d-c40d-43d9-8965-e85732ee6968-kube-api-access-ffkwh\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:00:20.953649 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:20.953645    2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b9ce846d-c40d-43d9-8965-e85732ee6968-config-file\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:00:21.538891 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:21.538803    2568 generic.go:356] "Generic (PLEG): container finished" podID="b9ce846d-c40d-43d9-8965-e85732ee6968" containerID="23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef" exitCode=0
Jul 24 23:00:21.538891 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:21.538854    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g"
Jul 24 23:00:21.539406 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:21.538887    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g" event={"ID":"b9ce846d-c40d-43d9-8965-e85732ee6968","Type":"ContainerDied","Data":"23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef"}
Jul 24 23:00:21.539406 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:21.538935    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6g98g" event={"ID":"b9ce846d-c40d-43d9-8965-e85732ee6968","Type":"ContainerDied","Data":"e693682e097c34dd25c3ee759ee43de5c0da32606c15e3d570b7007be3747412"}
Jul 24 23:00:21.539406 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:21.538958    2568 scope.go:117] "RemoveContainer" containerID="23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef"
Jul 24 23:00:21.547205 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:21.547187    2568 scope.go:117] "RemoveContainer" containerID="23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef"
Jul 24 23:00:21.547470 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:00:21.547453    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef\": container with ID starting with 23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef not found: ID does not exist" containerID="23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef"
Jul 24 23:00:21.547528 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:21.547478    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef"} err="failed to get container status \"23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef\": rpc error: code = NotFound desc = could not find container \"23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef\": container with ID starting with 23a8782e0f89b61f4ea64d5b963b55f098d2bbb98cba7b2e32413fbdf64d46ef not found: ID does not exist"
Jul 24 23:00:21.555413 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:21.555389    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6g98g"]
Jul 24 23:00:21.557076 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:21.557055    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6g98g"]
Jul 24 23:00:23.259241 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:23.259213    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9ce846d-c40d-43d9-8965-e85732ee6968" path="/var/lib/kubelet/pods/b9ce846d-c40d-43d9-8965-e85732ee6968/volumes"
Jul 24 23:00:24.898818 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.898790    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"]
Jul 24 23:00:24.899330 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.899293    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9ce846d-c40d-43d9-8965-e85732ee6968" containerName="limitador"
Jul 24 23:00:24.899330 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.899312    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9ce846d-c40d-43d9-8965-e85732ee6968" containerName="limitador"
Jul 24 23:00:24.899486 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.899424    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9ce846d-c40d-43d9-8965-e85732ee6968" containerName="limitador"
Jul 24 23:00:24.902210 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.902192    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"
Jul 24 23:00:24.904719 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.904689    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 24 23:00:24.904823 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.904693    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 23:00:24.905724 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.905709    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-b7l6h\""
Jul 24 23:00:24.905779 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.905732    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 23:00:24.911837 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.911815    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"]
Jul 24 23:00:24.985027 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.984993    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gznrd\" (UniqueName: \"kubernetes.io/projected/e9d4f6f6-a9e8-468d-8b8f-d52264498b3d-kube-api-access-gznrd\") pod \"postgres-69fb4594cf-c8l4w\" (UID: \"e9d4f6f6-a9e8-468d-8b8f-d52264498b3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"
Jul 24 23:00:24.985192 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:24.985088    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e9d4f6f6-a9e8-468d-8b8f-d52264498b3d-data\") pod \"postgres-69fb4594cf-c8l4w\" (UID: \"e9d4f6f6-a9e8-468d-8b8f-d52264498b3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"
Jul 24 23:00:25.086143 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:25.086096    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gznrd\" (UniqueName: \"kubernetes.io/projected/e9d4f6f6-a9e8-468d-8b8f-d52264498b3d-kube-api-access-gznrd\") pod \"postgres-69fb4594cf-c8l4w\" (UID: \"e9d4f6f6-a9e8-468d-8b8f-d52264498b3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"
Jul 24 23:00:25.086316 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:25.086202    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e9d4f6f6-a9e8-468d-8b8f-d52264498b3d-data\") pod \"postgres-69fb4594cf-c8l4w\" (UID: \"e9d4f6f6-a9e8-468d-8b8f-d52264498b3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"
Jul 24 23:00:25.086548 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:25.086532    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e9d4f6f6-a9e8-468d-8b8f-d52264498b3d-data\") pod \"postgres-69fb4594cf-c8l4w\" (UID: \"e9d4f6f6-a9e8-468d-8b8f-d52264498b3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"
Jul 24 23:00:25.094812 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:25.094794    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gznrd\" (UniqueName: \"kubernetes.io/projected/e9d4f6f6-a9e8-468d-8b8f-d52264498b3d-kube-api-access-gznrd\") pod \"postgres-69fb4594cf-c8l4w\" (UID: \"e9d4f6f6-a9e8-468d-8b8f-d52264498b3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"
Jul 24 23:00:25.213996 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:25.213967    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"
Jul 24 23:00:25.342630 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:25.342516    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"]
Jul 24 23:00:25.344921 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:00:25.344898    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9d4f6f6_a9e8_468d_8b8f_d52264498b3d.slice/crio-e17e067fb7bd86dda4994d794fefd081f2a5cc902db8a954a318a1b24d6c72bc WatchSource:0}: Error finding container e17e067fb7bd86dda4994d794fefd081f2a5cc902db8a954a318a1b24d6c72bc: Status 404 returned error can't find the container with id e17e067fb7bd86dda4994d794fefd081f2a5cc902db8a954a318a1b24d6c72bc
Jul 24 23:00:25.559473 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:25.559388    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w" event={"ID":"e9d4f6f6-a9e8-468d-8b8f-d52264498b3d","Type":"ContainerStarted","Data":"e17e067fb7bd86dda4994d794fefd081f2a5cc902db8a954a318a1b24d6c72bc"}
Jul 24 23:00:38.618752 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:38.618719    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w" event={"ID":"e9d4f6f6-a9e8-468d-8b8f-d52264498b3d","Type":"ContainerStarted","Data":"cc6b2eb2cf94f3190fda4e33009368e3423e4a71314c50381d6ab3a6c1d905ef"}
Jul 24 23:00:38.619155 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:38.618803    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"
Jul 24 23:00:38.635186 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:38.635145    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w" podStartSLOduration=1.866003281 podStartE2EDuration="14.635130199s" totalImagesPullingTime="12.769126918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:00:38.634033258 +0000 UTC m=+495.911066799" watchObservedRunningTime="2026-07-24 23:00:38.635130199 +0000 UTC m=+495.912163743"
Jul 24 23:00:44.653859 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:44.653829    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-c8l4w"
Jul 24 23:00:45.442786 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.442754    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-845cf8d598-hhmhc"]
Jul 24 23:00:45.444979 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.444964    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-845cf8d598-hhmhc"
Jul 24 23:00:45.452485 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.452461    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-845cf8d598-hhmhc"]
Jul 24 23:00:45.567783 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.567758    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkpdx\" (UniqueName: \"kubernetes.io/projected/36609dd9-3eb2-4c85-9e78-75b6e3ed4833-kube-api-access-bkpdx\") pod \"authorino-845cf8d598-hhmhc\" (UID: \"36609dd9-3eb2-4c85-9e78-75b6e3ed4833\") " pod="kuadrant-system/authorino-845cf8d598-hhmhc"
Jul 24 23:00:45.616659 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.616633    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-845cf8d598-hhmhc"]
Jul 24 23:00:45.616850 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:00:45.616833    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-bkpdx], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-845cf8d598-hhmhc" podUID="36609dd9-3eb2-4c85-9e78-75b6e3ed4833"
Jul 24 23:00:45.640363 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.640341    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f4d765f94-zp96c"]
Jul 24 23:00:45.642461 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.642446    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f4d765f94-zp96c"
Jul 24 23:00:45.644853 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.644839    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 24 23:00:45.650915 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.650893    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f4d765f94-zp96c"]
Jul 24 23:00:45.651031 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.650894    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-845cf8d598-hhmhc"
Jul 24 23:00:45.656461 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.656444    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-845cf8d598-hhmhc"
Jul 24 23:00:45.669069 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.669049    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkpdx\" (UniqueName: \"kubernetes.io/projected/36609dd9-3eb2-4c85-9e78-75b6e3ed4833-kube-api-access-bkpdx\") pod \"authorino-845cf8d598-hhmhc\" (UID: \"36609dd9-3eb2-4c85-9e78-75b6e3ed4833\") " pod="kuadrant-system/authorino-845cf8d598-hhmhc"
Jul 24 23:00:45.676955 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.676931    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkpdx\" (UniqueName: \"kubernetes.io/projected/36609dd9-3eb2-4c85-9e78-75b6e3ed4833-kube-api-access-bkpdx\") pod \"authorino-845cf8d598-hhmhc\" (UID: \"36609dd9-3eb2-4c85-9e78-75b6e3ed4833\") " pod="kuadrant-system/authorino-845cf8d598-hhmhc"
Jul 24 23:00:45.770168 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.770109    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkpdx\" (UniqueName: \"kubernetes.io/projected/36609dd9-3eb2-4c85-9e78-75b6e3ed4833-kube-api-access-bkpdx\") pod \"36609dd9-3eb2-4c85-9e78-75b6e3ed4833\" (UID: \"36609dd9-3eb2-4c85-9e78-75b6e3ed4833\") "
Jul 24 23:00:45.770319 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.770268    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-tls-cert\") pod \"authorino-6f4d765f94-zp96c\" (UID: \"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00\") " pod="kuadrant-system/authorino-6f4d765f94-zp96c"
Jul 24 23:00:45.770319 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.770311    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8sfk\" (UniqueName: \"kubernetes.io/projected/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-kube-api-access-d8sfk\") pod \"authorino-6f4d765f94-zp96c\" (UID: \"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00\") " pod="kuadrant-system/authorino-6f4d765f94-zp96c"
Jul 24 23:00:45.772057 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.772033    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36609dd9-3eb2-4c85-9e78-75b6e3ed4833-kube-api-access-bkpdx" (OuterVolumeSpecName: "kube-api-access-bkpdx") pod "36609dd9-3eb2-4c85-9e78-75b6e3ed4833" (UID: "36609dd9-3eb2-4c85-9e78-75b6e3ed4833"). InnerVolumeSpecName "kube-api-access-bkpdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:00:45.871152 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.871123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-tls-cert\") pod \"authorino-6f4d765f94-zp96c\" (UID: \"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00\") " pod="kuadrant-system/authorino-6f4d765f94-zp96c"
Jul 24 23:00:45.871295 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.871175    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8sfk\" (UniqueName: \"kubernetes.io/projected/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-kube-api-access-d8sfk\") pod \"authorino-6f4d765f94-zp96c\" (UID: \"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00\") " pod="kuadrant-system/authorino-6f4d765f94-zp96c"
Jul 24 23:00:45.871295 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.871231    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkpdx\" (UniqueName: \"kubernetes.io/projected/36609dd9-3eb2-4c85-9e78-75b6e3ed4833-kube-api-access-bkpdx\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:00:45.873376 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.873356    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-tls-cert\") pod \"authorino-6f4d765f94-zp96c\" (UID: \"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00\") " pod="kuadrant-system/authorino-6f4d765f94-zp96c"
Jul 24 23:00:45.878526 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.878503    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8sfk\" (UniqueName: \"kubernetes.io/projected/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-kube-api-access-d8sfk\") pod \"authorino-6f4d765f94-zp96c\" (UID: \"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00\") " pod="kuadrant-system/authorino-6f4d765f94-zp96c"
Jul 24 23:00:45.952175 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:45.952153    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f4d765f94-zp96c"
Jul 24 23:00:46.077346 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:46.077295    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f4d765f94-zp96c"]
Jul 24 23:00:46.079496 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:00:46.079470    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f06f0e1_7cff_4bbe_bb3c_9b941b15df00.slice/crio-7bd1cab9b81c5e0d5650582ec145cff89e83e808d03b4c60699a6076cf263c2c WatchSource:0}: Error finding container 7bd1cab9b81c5e0d5650582ec145cff89e83e808d03b4c60699a6076cf263c2c: Status 404 returned error can't find the container with id 7bd1cab9b81c5e0d5650582ec145cff89e83e808d03b4c60699a6076cf263c2c
Jul 24 23:00:46.658266 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:46.658230    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-845cf8d598-hhmhc"
Jul 24 23:00:46.658685 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:46.658291    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f4d765f94-zp96c" event={"ID":"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00","Type":"ContainerStarted","Data":"dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369"}
Jul 24 23:00:46.658685 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:46.658333    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f4d765f94-zp96c" event={"ID":"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00","Type":"ContainerStarted","Data":"7bd1cab9b81c5e0d5650582ec145cff89e83e808d03b4c60699a6076cf263c2c"}
Jul 24 23:00:46.675918 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:46.675850    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f4d765f94-zp96c" podStartSLOduration=1.202177955 podStartE2EDuration="1.675830323s" totalImagesPullingTime="473.652368ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:00:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:00:46.674800366 +0000 UTC m=+503.951833911" watchObservedRunningTime="2026-07-24 23:00:46.675830323 +0000 UTC m=+503.952863868"
Jul 24 23:00:46.697965 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:46.697940    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-845cf8d598-hhmhc"]
Jul 24 23:00:46.703106 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:46.703081    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rtmbg"]
Jul 24 23:00:46.703317 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:46.703292    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-rtmbg" podUID="33c79b50-87e9-4232-a31a-06cfeb02d53b" containerName="authorino" containerID="cri-o://fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99" gracePeriod=30
Jul 24 23:00:46.705515 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:46.705493    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-845cf8d598-hhmhc"]
Jul 24 23:00:46.953637 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:46.953611    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rtmbg"
Jul 24 23:00:47.081619 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.081527    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2lf4\" (UniqueName: \"kubernetes.io/projected/33c79b50-87e9-4232-a31a-06cfeb02d53b-kube-api-access-d2lf4\") pod \"33c79b50-87e9-4232-a31a-06cfeb02d53b\" (UID: \"33c79b50-87e9-4232-a31a-06cfeb02d53b\") "
Jul 24 23:00:47.083652 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.083619    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33c79b50-87e9-4232-a31a-06cfeb02d53b-kube-api-access-d2lf4" (OuterVolumeSpecName: "kube-api-access-d2lf4") pod "33c79b50-87e9-4232-a31a-06cfeb02d53b" (UID: "33c79b50-87e9-4232-a31a-06cfeb02d53b"). InnerVolumeSpecName "kube-api-access-d2lf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:00:47.182838 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.182805    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2lf4\" (UniqueName: \"kubernetes.io/projected/33c79b50-87e9-4232-a31a-06cfeb02d53b-kube-api-access-d2lf4\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:00:47.258670 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.258637    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36609dd9-3eb2-4c85-9e78-75b6e3ed4833" path="/var/lib/kubelet/pods/36609dd9-3eb2-4c85-9e78-75b6e3ed4833/volumes"
Jul 24 23:00:47.663668 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.663639    2568 generic.go:356] "Generic (PLEG): container finished" podID="33c79b50-87e9-4232-a31a-06cfeb02d53b" containerID="fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99" exitCode=0
Jul 24 23:00:47.664087 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.663682    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rtmbg"
Jul 24 23:00:47.664087 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.663727    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rtmbg" event={"ID":"33c79b50-87e9-4232-a31a-06cfeb02d53b","Type":"ContainerDied","Data":"fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99"}
Jul 24 23:00:47.664087 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.663770    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rtmbg" event={"ID":"33c79b50-87e9-4232-a31a-06cfeb02d53b","Type":"ContainerDied","Data":"ff0887a695bd8d97b0c6f37814d41f41fcb1a28938e9695ac174c1b5719710c1"}
Jul 24 23:00:47.664087 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.663791    2568 scope.go:117] "RemoveContainer" containerID="fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99"
Jul 24 23:00:47.672069 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.672052    2568 scope.go:117] "RemoveContainer" containerID="fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99"
Jul 24 23:00:47.672340 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:00:47.672317    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99\": container with ID starting with fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99 not found: ID does not exist" containerID="fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99"
Jul 24 23:00:47.672402 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.672354    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99"} err="failed to get container status \"fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99\": rpc error: code = NotFound desc = could not find container \"fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99\": container with ID starting with fd8267891e3bdbac64646c963851c715db67feba81561abd6972c9db85b6ad99 not found: ID does not exist"
Jul 24 23:00:47.681103 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.681083    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rtmbg"]
Jul 24 23:00:47.685532 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:47.685513    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rtmbg"]
Jul 24 23:00:49.257331 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:49.257294    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33c79b50-87e9-4232-a31a-06cfeb02d53b" path="/var/lib/kubelet/pods/33c79b50-87e9-4232-a31a-06cfeb02d53b/volumes"
Jul 24 23:00:51.306160 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.306126    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-dd7c4cdfc-sckwd"]
Jul 24 23:00:51.306572 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.306514    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33c79b50-87e9-4232-a31a-06cfeb02d53b" containerName="authorino"
Jul 24 23:00:51.306572 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.306527    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="33c79b50-87e9-4232-a31a-06cfeb02d53b" containerName="authorino"
Jul 24 23:00:51.306642 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.306610    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="33c79b50-87e9-4232-a31a-06cfeb02d53b" containerName="authorino"
Jul 24 23:00:51.309557 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.309540    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:00:51.312127 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.312101    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 23:00:51.312270 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.312193    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 23:00:51.312270 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.312192    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dq7r7\""
Jul 24 23:00:51.318068 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.318040    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-dd7c4cdfc-sckwd"]
Jul 24 23:00:51.418481 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.418445    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6nd7\" (UniqueName: \"kubernetes.io/projected/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-kube-api-access-b6nd7\") pod \"maas-controller-dd7c4cdfc-sckwd\" (UID: \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\") " pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:00:51.418664 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.418549    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-webhook-cert\") pod \"maas-controller-dd7c4cdfc-sckwd\" (UID: \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\") " pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:00:51.519980 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.519947    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6nd7\" (UniqueName: \"kubernetes.io/projected/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-kube-api-access-b6nd7\") pod \"maas-controller-dd7c4cdfc-sckwd\" (UID: \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\") " pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:00:51.520146 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.520045    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-webhook-cert\") pod \"maas-controller-dd7c4cdfc-sckwd\" (UID: \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\") " pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:00:51.520216 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:00:51.520199    2568 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 24 23:00:51.520302 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:00:51.520290    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-webhook-cert podName:077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc nodeName:}" failed. No retries permitted until 2026-07-24 23:00:52.020267046 +0000 UTC m=+509.297300572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-webhook-cert") pod "maas-controller-dd7c4cdfc-sckwd" (UID: "077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc") : secret "maas-controller-webhook-cert" not found
Jul 24 23:00:51.528799 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:51.528778    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6nd7\" (UniqueName: \"kubernetes.io/projected/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-kube-api-access-b6nd7\") pod \"maas-controller-dd7c4cdfc-sckwd\" (UID: \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\") " pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:00:52.023665 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:52.023627    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-webhook-cert\") pod \"maas-controller-dd7c4cdfc-sckwd\" (UID: \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\") " pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:00:52.026047 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:52.026015    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-webhook-cert\") pod \"maas-controller-dd7c4cdfc-sckwd\" (UID: \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\") " pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:00:52.220838 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:52.220810    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:00:52.356241 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:52.356158    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-dd7c4cdfc-sckwd"]
Jul 24 23:00:52.358705 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:00:52.358679    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077b3d5d_3e54_4ad1_b4cc_6f3e57391ebc.slice/crio-a85313584b7ec6f24bf8a7f1faa36c60ca9367ca857989d78f5aa0e65bedc176 WatchSource:0}: Error finding container a85313584b7ec6f24bf8a7f1faa36c60ca9367ca857989d78f5aa0e65bedc176: Status 404 returned error can't find the container with id a85313584b7ec6f24bf8a7f1faa36c60ca9367ca857989d78f5aa0e65bedc176
Jul 24 23:00:52.687836 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:52.687798    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd" event={"ID":"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc","Type":"ContainerStarted","Data":"a85313584b7ec6f24bf8a7f1faa36c60ca9367ca857989d78f5aa0e65bedc176"}
Jul 24 23:00:55.702473 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:55.702443    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd" event={"ID":"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc","Type":"ContainerStarted","Data":"553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499"}
Jul 24 23:00:55.702841 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:55.702660    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:00:55.720810 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:00:55.720769    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd" podStartSLOduration=2.206355491 podStartE2EDuration="4.720755792s" totalImagesPullingTime="2.514400301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:00:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:00:55.719474114 +0000 UTC m=+512.996507677" watchObservedRunningTime="2026-07-24 23:00:55.720755792 +0000 UTC m=+512.997789365"
Jul 24 23:01:06.709572 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:06.709538    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:01:07.421400 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.421365    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-dd7c4cdfc-sckwd"]
Jul 24 23:01:07.421599 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.421548    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd" podUID="077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc" containerName="manager" containerID="cri-o://553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499" gracePeriod=30
Jul 24 23:01:07.697017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.696995    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:01:07.757538 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.757508    2568 generic.go:356] "Generic (PLEG): container finished" podID="077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc" containerID="553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499" exitCode=0
Jul 24 23:01:07.757865 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.757553    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd"
Jul 24 23:01:07.757865 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.757603    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd" event={"ID":"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc","Type":"ContainerDied","Data":"553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499"}
Jul 24 23:01:07.757865 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.757652    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-dd7c4cdfc-sckwd" event={"ID":"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc","Type":"ContainerDied","Data":"a85313584b7ec6f24bf8a7f1faa36c60ca9367ca857989d78f5aa0e65bedc176"}
Jul 24 23:01:07.757865 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.757679    2568 scope.go:117] "RemoveContainer" containerID="553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499"
Jul 24 23:01:07.765804 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.765783    2568 scope.go:117] "RemoveContainer" containerID="553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499"
Jul 24 23:01:07.766030 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:01:07.766011    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499\": container with ID starting with 553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499 not found: ID does not exist" containerID="553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499"
Jul 24 23:01:07.766113 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.766037    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499"} err="failed to get container status \"553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499\": rpc error: code = NotFound desc = could not find container \"553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499\": container with ID starting with 553ab38e8f83199cdf72ac589e5ba79ea1079c9d9ca80625c6418dfa5daf4499 not found: ID does not exist"
Jul 24 23:01:07.774497 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.774478    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6nd7\" (UniqueName: \"kubernetes.io/projected/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-kube-api-access-b6nd7\") pod \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\" (UID: \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\") "
Jul 24 23:01:07.774589 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.774523    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-webhook-cert\") pod \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\" (UID: \"077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc\") "
Jul 24 23:01:07.776629 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.776601    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-kube-api-access-b6nd7" (OuterVolumeSpecName: "kube-api-access-b6nd7") pod "077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc" (UID: "077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc"). InnerVolumeSpecName "kube-api-access-b6nd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:01:07.776629 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.776614    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc" (UID: "077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:01:07.875511 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.875492    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6nd7\" (UniqueName: \"kubernetes.io/projected/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-kube-api-access-b6nd7\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:01:07.875511 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:07.875512    2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc-webhook-cert\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:01:08.078887 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:08.078858    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-dd7c4cdfc-sckwd"]
Jul 24 23:01:08.083385 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:08.083361    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-dd7c4cdfc-sckwd"]
Jul 24 23:01:09.256963 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:09.256932    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc" path="/var/lib/kubelet/pods/077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc/volumes"
Jul 24 23:01:28.647569 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.647538    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"]
Jul 24 23:01:28.647986 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.647883    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc" containerName="manager"
Jul 24 23:01:28.647986 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.647895    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc" containerName="manager"
Jul 24 23:01:28.647986 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.647946    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="077b3d5d-3e54-4ad1-b4cc-6f3e57391ebc" containerName="manager"
Jul 24 23:01:28.654079 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.654061    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:28.656644 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.656625    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 24 23:01:28.656723 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.656679    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-49q74\""
Jul 24 23:01:28.661727 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.661704    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"]
Jul 24 23:01:28.744106 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.744072    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd80415-b33f-4ced-9273-f590032afcc1-maas-api-tls\") pod \"maas-api-7d9754897c-lg5pg\" (UID: \"8fd80415-b33f-4ced-9273-f590032afcc1\") " pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:28.744236 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.744130    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfvmh\" (UniqueName: \"kubernetes.io/projected/8fd80415-b33f-4ced-9273-f590032afcc1-kube-api-access-gfvmh\") pod \"maas-api-7d9754897c-lg5pg\" (UID: \"8fd80415-b33f-4ced-9273-f590032afcc1\") " pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:28.744236 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.744208    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd80415-b33f-4ced-9273-f590032afcc1-openshift-service-ca\") pod \"maas-api-7d9754897c-lg5pg\" (UID: \"8fd80415-b33f-4ced-9273-f590032afcc1\") " pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:28.845397 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.845365    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfvmh\" (UniqueName: \"kubernetes.io/projected/8fd80415-b33f-4ced-9273-f590032afcc1-kube-api-access-gfvmh\") pod \"maas-api-7d9754897c-lg5pg\" (UID: \"8fd80415-b33f-4ced-9273-f590032afcc1\") " pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:28.845542 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.845440    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd80415-b33f-4ced-9273-f590032afcc1-openshift-service-ca\") pod \"maas-api-7d9754897c-lg5pg\" (UID: \"8fd80415-b33f-4ced-9273-f590032afcc1\") " pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:28.845542 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.845477    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd80415-b33f-4ced-9273-f590032afcc1-maas-api-tls\") pod \"maas-api-7d9754897c-lg5pg\" (UID: \"8fd80415-b33f-4ced-9273-f590032afcc1\") " pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:28.846194 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.846169    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd80415-b33f-4ced-9273-f590032afcc1-openshift-service-ca\") pod \"maas-api-7d9754897c-lg5pg\" (UID: \"8fd80415-b33f-4ced-9273-f590032afcc1\") " pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:28.848239 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.848216    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd80415-b33f-4ced-9273-f590032afcc1-maas-api-tls\") pod \"maas-api-7d9754897c-lg5pg\" (UID: \"8fd80415-b33f-4ced-9273-f590032afcc1\") " pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:28.856897 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.856875    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfvmh\" (UniqueName: \"kubernetes.io/projected/8fd80415-b33f-4ced-9273-f590032afcc1-kube-api-access-gfvmh\") pod \"maas-api-7d9754897c-lg5pg\" (UID: \"8fd80415-b33f-4ced-9273-f590032afcc1\") " pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:28.965403 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:28.965376    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:29.097935 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:29.097900    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"]
Jul 24 23:01:29.099984 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:01:29.099959    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd80415_b33f_4ced_9273_f590032afcc1.slice/crio-ea35cc42cd101027cbc7dcd69194c1af40f7aadf87e2260cd85364deca256381 WatchSource:0}: Error finding container ea35cc42cd101027cbc7dcd69194c1af40f7aadf87e2260cd85364deca256381: Status 404 returned error can't find the container with id ea35cc42cd101027cbc7dcd69194c1af40f7aadf87e2260cd85364deca256381
Jul 24 23:01:29.855642 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:29.855603    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg" event={"ID":"8fd80415-b33f-4ced-9273-f590032afcc1","Type":"ContainerStarted","Data":"ea35cc42cd101027cbc7dcd69194c1af40f7aadf87e2260cd85364deca256381"}
Jul 24 23:01:30.862533 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:30.862499    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg" event={"ID":"8fd80415-b33f-4ced-9273-f590032afcc1","Type":"ContainerStarted","Data":"7c79ff9c93b50da95c92611640f998888cd06c70e069c0e8c86f6d033b2ac8db"}
Jul 24 23:01:30.862997 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:30.862629    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:01:30.878448 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:30.878403    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg" podStartSLOduration=1.365704295 podStartE2EDuration="2.878387076s" totalImagesPullingTime="1.512682781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:01:30.878092653 +0000 UTC m=+548.155126221" watchObservedRunningTime="2026-07-24 23:01:30.878387076 +0000 UTC m=+548.155420620"
Jul 24 23:01:36.872343 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:01:36.872315    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7d9754897c-lg5pg"
Jul 24 23:02:04.181479 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.181443    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"]
Jul 24 23:02:04.185101 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.185080    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.187845 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.187824    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 23:02:04.187962 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.187825    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q7cqg\""
Jul 24 23:02:04.187962 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.187902    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 24 23:02:04.187962 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.187826    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 23:02:04.193573 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.193167    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"]
Jul 24 23:02:04.261507 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.261472    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.261507 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.261508    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jqjs\" (UniqueName: \"kubernetes.io/projected/24636d55-feb3-4eaf-9cfb-86312f4e8e70-kube-api-access-7jqjs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.261706 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.261545    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.261706 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.261592    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24636d55-feb3-4eaf-9cfb-86312f4e8e70-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.261706 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.261627    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.261706 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.261647    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.362788 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.362753    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jqjs\" (UniqueName: \"kubernetes.io/projected/24636d55-feb3-4eaf-9cfb-86312f4e8e70-kube-api-access-7jqjs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.362969 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.362808    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.363043 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.362974    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24636d55-feb3-4eaf-9cfb-86312f4e8e70-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.363113 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.363090    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.363167 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.363140    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.363217 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.363172    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.363302 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.363238    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.363396 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.363374    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.363511 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.363493    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.365262 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.365222    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24636d55-feb3-4eaf-9cfb-86312f4e8e70-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.365424 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.365403    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24636d55-feb3-4eaf-9cfb-86312f4e8e70-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.370032 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.370008    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jqjs\" (UniqueName: \"kubernetes.io/projected/24636d55-feb3-4eaf-9cfb-86312f4e8e70-kube-api-access-7jqjs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vst4v\" (UID: \"24636d55-feb3-4eaf-9cfb-86312f4e8e70\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.497088 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.497059    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:04.633394 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:04.633369    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"]
Jul 24 23:02:04.635338 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:02:04.635309    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24636d55_feb3_4eaf_9cfb_86312f4e8e70.slice/crio-095156d6cd2910f4a1dab60670f659db09955ae5f5fd9b0ae985be495d9bf623 WatchSource:0}: Error finding container 095156d6cd2910f4a1dab60670f659db09955ae5f5fd9b0ae985be495d9bf623: Status 404 returned error can't find the container with id 095156d6cd2910f4a1dab60670f659db09955ae5f5fd9b0ae985be495d9bf623
Jul 24 23:02:05.021345 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:05.021299    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v" event={"ID":"24636d55-feb3-4eaf-9cfb-86312f4e8e70","Type":"ContainerStarted","Data":"095156d6cd2910f4a1dab60670f659db09955ae5f5fd9b0ae985be495d9bf623"}
Jul 24 23:02:06.026937 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:06.026904    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v" event={"ID":"24636d55-feb3-4eaf-9cfb-86312f4e8e70","Type":"ContainerStarted","Data":"74354acbaa09de936200270e131f878aec7bbe515b7c5a2f4e1a57a4d6cd93fa"}
Jul 24 23:02:06.045547 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:06.045491    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v" podStartSLOduration=1.013575342 podStartE2EDuration="2.045471994s" totalImagesPullingTime="1.031896652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:02:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:02:06.043587046 +0000 UTC m=+583.320620590" watchObservedRunningTime="2026-07-24 23:02:06.045471994 +0000 UTC m=+583.322505537"
Jul 24 23:02:11.497262 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.497222    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"]
Jul 24 23:02:11.499963 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.499943    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.502377 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.502355    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 24 23:02:11.510579 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.510558    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"]
Jul 24 23:02:11.631770 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.631732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.631949 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.631814    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vfwc\" (UniqueName: \"kubernetes.io/projected/39d98e22-18bc-4df2-a552-21471d2a3ba7-kube-api-access-5vfwc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.631949 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.631935    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.632049 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.631964    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.632049 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.631995    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.632049 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.632030    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39d98e22-18bc-4df2-a552-21471d2a3ba7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.733001 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.732963    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.733001 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.733007    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.733296 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.733036    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.733296 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.733079    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39d98e22-18bc-4df2-a552-21471d2a3ba7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.733296 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.733129    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.733296 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.733168    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vfwc\" (UniqueName: \"kubernetes.io/projected/39d98e22-18bc-4df2-a552-21471d2a3ba7-kube-api-access-5vfwc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.733495 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.733428    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.733495 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.733465    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.733586 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.733531    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.735241 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.735218    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39d98e22-18bc-4df2-a552-21471d2a3ba7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.735523 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.735506    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39d98e22-18bc-4df2-a552-21471d2a3ba7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.740946 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.740922    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vfwc\" (UniqueName: \"kubernetes.io/projected/39d98e22-18bc-4df2-a552-21471d2a3ba7-kube-api-access-5vfwc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kbfr\" (UID: \"39d98e22-18bc-4df2-a552-21471d2a3ba7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.810092 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.810035    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:11.938548 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:11.938519    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"]
Jul 24 23:02:11.941759 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:02:11.941733    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39d98e22_18bc_4df2_a552_21471d2a3ba7.slice/crio-34476fa04f6301bd784493cd60fe08a277a1113a7babbf297d2ce7092f85afd3 WatchSource:0}: Error finding container 34476fa04f6301bd784493cd60fe08a277a1113a7babbf297d2ce7092f85afd3: Status 404 returned error can't find the container with id 34476fa04f6301bd784493cd60fe08a277a1113a7babbf297d2ce7092f85afd3
Jul 24 23:02:12.053826 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:12.053799    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr" event={"ID":"39d98e22-18bc-4df2-a552-21471d2a3ba7","Type":"ContainerStarted","Data":"34476fa04f6301bd784493cd60fe08a277a1113a7babbf297d2ce7092f85afd3"}
Jul 24 23:02:13.059142 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:13.059108    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr" event={"ID":"39d98e22-18bc-4df2-a552-21471d2a3ba7","Type":"ContainerStarted","Data":"1e54151914b3fb89a4e0b45c538820949b306c2eba7f41abde1d133bd6e14d4f"}
Jul 24 23:02:13.077146 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:13.077097    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr" podStartSLOduration=1.901297444 podStartE2EDuration="2.077079804s" totalImagesPullingTime="175.78236ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:02:13.075113039 +0000 UTC m=+590.352146584" watchObservedRunningTime="2026-07-24 23:02:13.077079804 +0000 UTC m=+590.354113348"
Jul 24 23:02:14.497866 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:14.497831    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:14.498388 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:14.497910    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:14.510799 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:14.510769    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:15.078953 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:15.078921    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vst4v"
Jul 24 23:02:21.810342 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:21.810306    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:21.810807 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:21.810371    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:21.822579 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:21.822552    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:22.118806 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:22.118729    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kbfr"
Jul 24 23:02:36.193116 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.193078    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"]
Jul 24 23:02:36.200587 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.200555    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.203200 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.203176    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 24 23:02:36.210037 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.210013    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"]
Jul 24 23:02:36.259905 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.259876    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.260086 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.259911    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0daa5fc8-079b-4c42-a702-b3de326ff7a2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.260086 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.259957    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4shzs\" (UniqueName: \"kubernetes.io/projected/0daa5fc8-079b-4c42-a702-b3de326ff7a2-kube-api-access-4shzs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.260086 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.260045    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.260086 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.260070    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.260243 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.260113    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.360629 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.360591    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.360822 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.360638    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.360822 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.360670    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.360822 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.360701    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.360822 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.360721    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0daa5fc8-079b-4c42-a702-b3de326ff7a2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.360822 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.360808    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4shzs\" (UniqueName: \"kubernetes.io/projected/0daa5fc8-079b-4c42-a702-b3de326ff7a2-kube-api-access-4shzs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.361142 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.361119    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.361305 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.361140    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.361386 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.361177    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.363058 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.363033    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0daa5fc8-079b-4c42-a702-b3de326ff7a2-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.363430 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.363406    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0daa5fc8-079b-4c42-a702-b3de326ff7a2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.368324 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.368301    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4shzs\" (UniqueName: \"kubernetes.io/projected/0daa5fc8-079b-4c42-a702-b3de326ff7a2-kube-api-access-4shzs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-85ldr\" (UID: \"0daa5fc8-079b-4c42-a702-b3de326ff7a2\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.513047 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.512955    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:36.661919 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:36.661895    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"]
Jul 24 23:02:36.664483 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:02:36.664452    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0daa5fc8_079b_4c42_a702_b3de326ff7a2.slice/crio-7815b4dc94a93029dcafacc709d3cfbbd1534d353eb74755a778bbe54c3a9c9d WatchSource:0}: Error finding container 7815b4dc94a93029dcafacc709d3cfbbd1534d353eb74755a778bbe54c3a9c9d: Status 404 returned error can't find the container with id 7815b4dc94a93029dcafacc709d3cfbbd1534d353eb74755a778bbe54c3a9c9d
Jul 24 23:02:37.173924 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:37.173837    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr" event={"ID":"0daa5fc8-079b-4c42-a702-b3de326ff7a2","Type":"ContainerStarted","Data":"fb67b9a08f0eb37231efb7f12f81081ba105bde3c144c9872eb2c0af71622ace"}
Jul 24 23:02:37.173924 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:37.173886    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr" event={"ID":"0daa5fc8-079b-4c42-a702-b3de326ff7a2","Type":"ContainerStarted","Data":"7815b4dc94a93029dcafacc709d3cfbbd1534d353eb74755a778bbe54c3a9c9d"}
Jul 24 23:02:37.191486 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:37.191427    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr" podStartSLOduration=1.031775223 podStartE2EDuration="1.19140645s" totalImagesPullingTime="159.631227ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:02:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:02:37.190287238 +0000 UTC m=+614.467320798" watchObservedRunningTime="2026-07-24 23:02:37.19140645 +0000 UTC m=+614.468439994"
Jul 24 23:02:46.513158 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:46.513071    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:46.513158 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:46.513123    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:46.525160 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:46.525140    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:02:47.230577 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:02:47.230545    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-85ldr"
Jul 24 23:03:32.335315 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:32.335272    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f4d765f94-zp96c"]
Jul 24 23:03:32.335904 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:32.335507    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6f4d765f94-zp96c" podUID="2f06f0e1-7cff-4bbe-bb3c-9b941b15df00" containerName="authorino" containerID="cri-o://dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369" gracePeriod=30
Jul 24 23:03:32.585112 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:32.585087    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f4d765f94-zp96c"
Jul 24 23:03:32.677951 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:32.677866    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8sfk\" (UniqueName: \"kubernetes.io/projected/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-kube-api-access-d8sfk\") pod \"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00\" (UID: \"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00\") "
Jul 24 23:03:32.677951 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:32.677933    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-tls-cert\") pod \"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00\" (UID: \"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00\") "
Jul 24 23:03:32.679967 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:32.679935    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-kube-api-access-d8sfk" (OuterVolumeSpecName: "kube-api-access-d8sfk") pod "2f06f0e1-7cff-4bbe-bb3c-9b941b15df00" (UID: "2f06f0e1-7cff-4bbe-bb3c-9b941b15df00"). InnerVolumeSpecName "kube-api-access-d8sfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:03:32.687951 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:32.687923    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2f06f0e1-7cff-4bbe-bb3c-9b941b15df00" (UID: "2f06f0e1-7cff-4bbe-bb3c-9b941b15df00"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:03:32.779039 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:32.779007    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8sfk\" (UniqueName: \"kubernetes.io/projected/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-kube-api-access-d8sfk\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:03:32.779039 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:32.779034    2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00-tls-cert\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:03:33.421066 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:33.421031    2568 generic.go:356] "Generic (PLEG): container finished" podID="2f06f0e1-7cff-4bbe-bb3c-9b941b15df00" containerID="dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369" exitCode=0
Jul 24 23:03:33.421489 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:33.421090    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f4d765f94-zp96c"
Jul 24 23:03:33.421489 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:33.421117    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f4d765f94-zp96c" event={"ID":"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00","Type":"ContainerDied","Data":"dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369"}
Jul 24 23:03:33.421489 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:33.421155    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f4d765f94-zp96c" event={"ID":"2f06f0e1-7cff-4bbe-bb3c-9b941b15df00","Type":"ContainerDied","Data":"7bd1cab9b81c5e0d5650582ec145cff89e83e808d03b4c60699a6076cf263c2c"}
Jul 24 23:03:33.421489 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:33.421172    2568 scope.go:117] "RemoveContainer" containerID="dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369"
Jul 24 23:03:33.429619 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:33.429602    2568 scope.go:117] "RemoveContainer" containerID="dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369"
Jul 24 23:03:33.429872 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:03:33.429853    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369\": container with ID starting with dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369 not found: ID does not exist" containerID="dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369"
Jul 24 23:03:33.429934 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:33.429880    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369"} err="failed to get container status \"dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369\": rpc error: code = NotFound desc = could not find container \"dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369\": container with ID starting with dba5b4fd1fb3ea2f5c39209ddae99b57e3a7e89be43e7485920b04658deab369 not found: ID does not exist"
Jul 24 23:03:33.440601 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:33.440576    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f4d765f94-zp96c"]
Jul 24 23:03:33.443531 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:33.443512    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f4d765f94-zp96c"]
Jul 24 23:03:35.257696 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:03:35.257669    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f06f0e1-7cff-4bbe-bb3c-9b941b15df00" path="/var/lib/kubelet/pods/2f06f0e1-7cff-4bbe-bb3c-9b941b15df00/volumes"
Jul 24 23:14:07.279917 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.279878    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"]
Jul 24 23:14:07.280533 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.280312    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f06f0e1-7cff-4bbe-bb3c-9b941b15df00" containerName="authorino"
Jul 24 23:14:07.280533 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.280328    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f06f0e1-7cff-4bbe-bb3c-9b941b15df00" containerName="authorino"
Jul 24 23:14:07.280533 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.280462    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f06f0e1-7cff-4bbe-bb3c-9b941b15df00" containerName="authorino"
Jul 24 23:14:07.283551 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.283530    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.285948 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.285929    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-f3bee56b-kserve-self-signed-certs\""
Jul 24 23:14:07.297720 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.297700    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"]
Jul 24 23:14:07.317668 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.317646    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-home\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.317759 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.317670    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-dshm\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.317759 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.317692    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tls-certs\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.317759 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.317708    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4gjq\" (UniqueName: \"kubernetes.io/projected/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-kube-api-access-p4gjq\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.317872 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.317780    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tmp-dir\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.317872 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.317827    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-model-cache\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.418680 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.418647    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-home\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.418680 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.418681    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-dshm\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.418871 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.418699    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tls-certs\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.418871 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.418721    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4gjq\" (UniqueName: \"kubernetes.io/projected/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-kube-api-access-p4gjq\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.418871 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.418778    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tmp-dir\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.418871 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.418835    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-model-cache\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.419051 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.419018    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-home\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.419211 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.419189    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tmp-dir\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.419280 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.419200    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-model-cache\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.420894 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.420877    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-dshm\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.421129 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.421112    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tls-certs\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.426617 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.426598    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4gjq\" (UniqueName: \"kubernetes.io/projected/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-kube-api-access-p4gjq\") pod \"e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") " pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.592885 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.592808    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:07.719471 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:07.719440    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"]
Jul 24 23:14:07.721887 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:14:07.721859    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod653b56cc_1afe_4d77_b7dd_8e0ec11ef6db.slice/crio-3e1c055f903ff19aebffa052f9fbb90daffd076385d562007c86b34b101286e1 WatchSource:0}: Error finding container 3e1c055f903ff19aebffa052f9fbb90daffd076385d562007c86b34b101286e1: Status 404 returned error can't find the container with id 3e1c055f903ff19aebffa052f9fbb90daffd076385d562007c86b34b101286e1
Jul 24 23:14:08.293537 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:08.293505    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2" event={"ID":"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db","Type":"ContainerStarted","Data":"cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce"}
Jul 24 23:14:08.293914 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:08.293548    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2" event={"ID":"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db","Type":"ContainerStarted","Data":"3e1c055f903ff19aebffa052f9fbb90daffd076385d562007c86b34b101286e1"}
Jul 24 23:14:08.310895 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:08.310848    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2" podStartSLOduration=1.310832627 podStartE2EDuration="1.310832627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:14:08.309857214 +0000 UTC m=+1305.586890758" watchObservedRunningTime="2026-07-24 23:14:08.310832627 +0000 UTC m=+1305.587866214"
Jul 24 23:14:17.593576 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:17.593476    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:17.593576 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:17.593551    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:17.605490 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:17.605471    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:18.347018 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:18.346989    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:14:27.883995 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:27.883964    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"]
Jul 24 23:14:27.887288 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:27.887242    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:27.889712 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:27.889693    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-f3bee56b-kserve-self-signed-certs\""
Jul 24 23:14:27.897240 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:27.897219    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"]
Jul 24 23:14:27.999482 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:27.999455    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tls-certs\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:27.999624 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:27.999497    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-dshm\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:27.999624 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:27.999584    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-home\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:27.999731 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:27.999641    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lplf4\" (UniqueName: \"kubernetes.io/projected/de8037ee-b4f2-44a0-bffa-8b476fb366e2-kube-api-access-lplf4\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:27.999731 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:27.999669    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-model-cache\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:27.999731 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:27.999701    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tmp-dir\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.100514 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.100475    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-dshm\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.100654 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.100534    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-home\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.100654 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.100555    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lplf4\" (UniqueName: \"kubernetes.io/projected/de8037ee-b4f2-44a0-bffa-8b476fb366e2-kube-api-access-lplf4\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.100654 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.100585    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-model-cache\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.100654 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.100611    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tmp-dir\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.100654 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.100631    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tls-certs\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.100953 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.100934    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-home\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.101001 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.100970    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-model-cache\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.101064 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.100997    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tmp-dir\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.102587 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.102557    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-dshm\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.102892 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.102876    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tls-certs\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.107983 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.107960    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lplf4\" (UniqueName: \"kubernetes.io/projected/de8037ee-b4f2-44a0-bffa-8b476fb366e2-kube-api-access-lplf4\") pod \"e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") " pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.195838 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.195808    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:28.322837 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.322814    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"]
Jul 24 23:14:28.324348 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:14:28.324319    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde8037ee_b4f2_44a0_bffa_8b476fb366e2.slice/crio-24aecf135d65795dd42658f90974b63b62e8f28e661f8b49211ef5f76d2964b3 WatchSource:0}: Error finding container 24aecf135d65795dd42658f90974b63b62e8f28e661f8b49211ef5f76d2964b3: Status 404 returned error can't find the container with id 24aecf135d65795dd42658f90974b63b62e8f28e661f8b49211ef5f76d2964b3
Jul 24 23:14:28.379222 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:28.379197    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq" event={"ID":"de8037ee-b4f2-44a0-bffa-8b476fb366e2","Type":"ContainerStarted","Data":"24aecf135d65795dd42658f90974b63b62e8f28e661f8b49211ef5f76d2964b3"}
Jul 24 23:14:29.384874 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:29.384842    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq" event={"ID":"de8037ee-b4f2-44a0-bffa-8b476fb366e2","Type":"ContainerStarted","Data":"149da260a155405ec6bb3f29ed11c532e5d667ab6f82327d4b1f0484b7c55218"}
Jul 24 23:14:29.401864 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:29.401817    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq" podStartSLOduration=2.401799351 podStartE2EDuration="2.401799351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:14:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:14:29.399844585 +0000 UTC m=+1326.676878138" watchObservedRunningTime="2026-07-24 23:14:29.401799351 +0000 UTC m=+1326.678832913"
Jul 24 23:14:38.196410 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:38.196374    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:38.196410 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:38.196416    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:38.208531 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:38.208502    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:14:38.432657 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:14:38.432627    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:15:12.763611 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:12.763534    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"]
Jul 24 23:15:13.490610 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:13.490583    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"]
Jul 24 23:15:27.781949 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:27.781886    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2" podUID="653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" containerName="main" containerID="cri-o://cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce" gracePeriod=45
Jul 24 23:15:28.033941 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.033889    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:15:28.046435 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.046414    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-dshm\") pod \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") "
Jul 24 23:15:28.046561 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.046445    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tls-certs\") pod \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") "
Jul 24 23:15:28.046561 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.046476    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-model-cache\") pod \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") "
Jul 24 23:15:28.046561 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.046512    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-home\") pod \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") "
Jul 24 23:15:28.046561 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.046547    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4gjq\" (UniqueName: \"kubernetes.io/projected/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-kube-api-access-p4gjq\") pod \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") "
Jul 24 23:15:28.046755 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.046613    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tmp-dir\") pod \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\" (UID: \"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db\") "
Jul 24 23:15:28.046806 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.046764    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-model-cache" (OuterVolumeSpecName: "model-cache") pod "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" (UID: "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:15:28.046806 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.046789    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-home" (OuterVolumeSpecName: "home") pod "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" (UID: "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:15:28.046994 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.046965    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" (UID: "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:15:28.047065 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.047047    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-model-cache\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.047120 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.047070    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-home\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.048664 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.048636    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-dshm" (OuterVolumeSpecName: "dshm") pod "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" (UID: "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:15:28.048760 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.048668    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-kube-api-access-p4gjq" (OuterVolumeSpecName: "kube-api-access-p4gjq") pod "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" (UID: "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db"). InnerVolumeSpecName "kube-api-access-p4gjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:15:28.048760 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.048720    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" (UID: "653b56cc-1afe-4d77-b7dd-8e0ec11ef6db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:15:28.148164 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.148129    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-dshm\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.148164 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.148155    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tls-certs\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.148164 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.148164    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4gjq\" (UniqueName: \"kubernetes.io/projected/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-kube-api-access-p4gjq\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.148164 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.148173    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db-tmp-dir\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.509173 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.509129    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq" podUID="de8037ee-b4f2-44a0-bffa-8b476fb366e2" containerName="main" containerID="cri-o://149da260a155405ec6bb3f29ed11c532e5d667ab6f82327d4b1f0484b7c55218" gracePeriod=45
Jul 24 23:15:28.656189 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.656161    2568 generic.go:356] "Generic (PLEG): container finished" podID="653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" containerID="cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce" exitCode=0
Jul 24 23:15:28.656385 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.656207    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"
Jul 24 23:15:28.656385 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.656239    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2" event={"ID":"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db","Type":"ContainerDied","Data":"cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce"}
Jul 24 23:15:28.656385 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.656297    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2" event={"ID":"653b56cc-1afe-4d77-b7dd-8e0ec11ef6db","Type":"ContainerDied","Data":"3e1c055f903ff19aebffa052f9fbb90daffd076385d562007c86b34b101286e1"}
Jul 24 23:15:28.656385 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.656320    2568 scope.go:117] "RemoveContainer" containerID="cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce"
Jul 24 23:15:28.658070 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.658038    2568 generic.go:356] "Generic (PLEG): container finished" podID="de8037ee-b4f2-44a0-bffa-8b476fb366e2" containerID="149da260a155405ec6bb3f29ed11c532e5d667ab6f82327d4b1f0484b7c55218" exitCode=0
Jul 24 23:15:28.658164 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.658111    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq" event={"ID":"de8037ee-b4f2-44a0-bffa-8b476fb366e2","Type":"ContainerDied","Data":"149da260a155405ec6bb3f29ed11c532e5d667ab6f82327d4b1f0484b7c55218"}
Jul 24 23:15:28.665161 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.665038    2568 scope.go:117] "RemoveContainer" containerID="cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce"
Jul 24 23:15:28.668455 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:15:28.668424    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce\": container with ID starting with cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce not found: ID does not exist" containerID="cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce"
Jul 24 23:15:28.668559 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.668463    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce"} err="failed to get container status \"cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce\": rpc error: code = NotFound desc = could not find container \"cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce\": container with ID starting with cc6aede5eb60bb6b9480bd13ac7003f63611028a609df68148bcd8ba43f8d5ce not found: ID does not exist"
Jul 24 23:15:28.677465 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.677355    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"]
Jul 24 23:15:28.679864 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.679846    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-f3bee56b-kserve-79d595968c-b94w2"]
Jul 24 23:15:28.745016 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.744995    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:15:28.753825 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.753808    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lplf4\" (UniqueName: \"kubernetes.io/projected/de8037ee-b4f2-44a0-bffa-8b476fb366e2-kube-api-access-lplf4\") pod \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") "
Jul 24 23:15:28.753887 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.753854    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tmp-dir\") pod \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") "
Jul 24 23:15:28.753923 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.753888    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tls-certs\") pod \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") "
Jul 24 23:15:28.753965 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.753938    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-home\") pod \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") "
Jul 24 23:15:28.754022 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.753963    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-dshm\") pod \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") "
Jul 24 23:15:28.754022 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.753993    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-model-cache\") pod \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\" (UID: \"de8037ee-b4f2-44a0-bffa-8b476fb366e2\") "
Jul 24 23:15:28.754161 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.754136    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de8037ee-b4f2-44a0-bffa-8b476fb366e2" (UID: "de8037ee-b4f2-44a0-bffa-8b476fb366e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:15:28.754161 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.754148    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-home" (OuterVolumeSpecName: "home") pod "de8037ee-b4f2-44a0-bffa-8b476fb366e2" (UID: "de8037ee-b4f2-44a0-bffa-8b476fb366e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:15:28.754324 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.754307    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-home\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.754385 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.754336    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tmp-dir\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.754385 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.754305    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "de8037ee-b4f2-44a0-bffa-8b476fb366e2" (UID: "de8037ee-b4f2-44a0-bffa-8b476fb366e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:15:28.755915 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.755897    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-dshm" (OuterVolumeSpecName: "dshm") pod "de8037ee-b4f2-44a0-bffa-8b476fb366e2" (UID: "de8037ee-b4f2-44a0-bffa-8b476fb366e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 23:15:28.756160 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.755924    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de8037ee-b4f2-44a0-bffa-8b476fb366e2-kube-api-access-lplf4" (OuterVolumeSpecName: "kube-api-access-lplf4") pod "de8037ee-b4f2-44a0-bffa-8b476fb366e2" (UID: "de8037ee-b4f2-44a0-bffa-8b476fb366e2"). InnerVolumeSpecName "kube-api-access-lplf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:15:28.756160 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.755969    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de8037ee-b4f2-44a0-bffa-8b476fb366e2" (UID: "de8037ee-b4f2-44a0-bffa-8b476fb366e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:15:28.855161 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.855135    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-dshm\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.855161 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.855158    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de8037ee-b4f2-44a0-bffa-8b476fb366e2-model-cache\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.855161 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.855169    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lplf4\" (UniqueName: \"kubernetes.io/projected/de8037ee-b4f2-44a0-bffa-8b476fb366e2-kube-api-access-lplf4\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:28.855605 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:28.855177    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de8037ee-b4f2-44a0-bffa-8b476fb366e2-tls-certs\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:15:29.257886 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:29.257857    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" path="/var/lib/kubelet/pods/653b56cc-1afe-4d77-b7dd-8e0ec11ef6db/volumes"
Jul 24 23:15:29.665447 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:29.665368    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"
Jul 24 23:15:29.665447 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:29.665394    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq" event={"ID":"de8037ee-b4f2-44a0-bffa-8b476fb366e2","Type":"ContainerDied","Data":"24aecf135d65795dd42658f90974b63b62e8f28e661f8b49211ef5f76d2964b3"}
Jul 24 23:15:29.665664 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:29.665448    2568 scope.go:117] "RemoveContainer" containerID="149da260a155405ec6bb3f29ed11c532e5d667ab6f82327d4b1f0484b7c55218"
Jul 24 23:15:29.681243 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:29.681220    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"]
Jul 24 23:15:29.687046 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:29.687026    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-f3bee56b-kserve-cd48dc867-msdfq"]
Jul 24 23:15:31.257123 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:15:31.257093    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de8037ee-b4f2-44a0-bffa-8b476fb366e2" path="/var/lib/kubelet/pods/de8037ee-b4f2-44a0-bffa-8b476fb366e2/volumes"
Jul 24 23:19:05.294881 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.294846    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"]
Jul 24 23:19:05.295535 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.295203    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" containerName="main"
Jul 24 23:19:05.295535 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.295220    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" containerName="main"
Jul 24 23:19:05.295535 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.295243    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de8037ee-b4f2-44a0-bffa-8b476fb366e2" containerName="main"
Jul 24 23:19:05.295535 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.295275    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="de8037ee-b4f2-44a0-bffa-8b476fb366e2" containerName="main"
Jul 24 23:19:05.295535 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.295388    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="de8037ee-b4f2-44a0-bffa-8b476fb366e2" containerName="main"
Jul 24 23:19:05.295535 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.295402    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="653b56cc-1afe-4d77-b7dd-8e0ec11ef6db" containerName="main"
Jul 24 23:19:05.298419 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.298398    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:05.301008 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.300988    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a599b375-serving-cert\""
Jul 24 23:19:05.307782 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.307759    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"]
Jul 24 23:19:05.388472 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.388431    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nftqc\" (UniqueName: \"kubernetes.io/projected/c648bad5-f6cb-4964-ae31-dc720f31dd8c-kube-api-access-nftqc\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:05.388672 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.388483    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c648bad5-f6cb-4964-ae31-dc720f31dd8c-openshift-service-ca\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:05.388672 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.388546    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:05.489706 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.489674    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nftqc\" (UniqueName: \"kubernetes.io/projected/c648bad5-f6cb-4964-ae31-dc720f31dd8c-kube-api-access-nftqc\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:05.489860 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.489714    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c648bad5-f6cb-4964-ae31-dc720f31dd8c-openshift-service-ca\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:05.489860 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.489732    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:05.489944 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:19:05.489871    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-serving-cert: secret "maas-api-e2e-ait-a599b375-serving-cert" not found
Jul 24 23:19:05.489944 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:19:05.489941    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls podName:c648bad5-f6cb-4964-ae31-dc720f31dd8c nodeName:}" failed. No retries permitted until 2026-07-24 23:19:05.98992586 +0000 UTC m=+1603.266959381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls") pod "maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm" (UID: "c648bad5-f6cb-4964-ae31-dc720f31dd8c") : secret "maas-api-e2e-ait-a599b375-serving-cert" not found
Jul 24 23:19:05.490307 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.490287    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c648bad5-f6cb-4964-ae31-dc720f31dd8c-openshift-service-ca\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:05.498164 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.498138    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nftqc\" (UniqueName: \"kubernetes.io/projected/c648bad5-f6cb-4964-ae31-dc720f31dd8c-kube-api-access-nftqc\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:05.994849 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:05.994819    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:05.995066 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:19:05.994972    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-serving-cert: secret "maas-api-e2e-ait-a599b375-serving-cert" not found
Jul 24 23:19:05.995066 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:19:05.995059    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls podName:c648bad5-f6cb-4964-ae31-dc720f31dd8c nodeName:}" failed. No retries permitted until 2026-07-24 23:19:06.995044095 +0000 UTC m=+1604.272077617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls") pod "maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm" (UID: "c648bad5-f6cb-4964-ae31-dc720f31dd8c") : secret "maas-api-e2e-ait-a599b375-serving-cert" not found
Jul 24 23:19:07.005762 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:07.005722    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:07.008082 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:07.008056    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls\") pod \"maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:07.108590 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:07.108562    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:07.441642 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:07.441618    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"]
Jul 24 23:19:07.443764 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:19:07.443737    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc648bad5_f6cb_4964_ae31_dc720f31dd8c.slice/crio-3ddf1c06d4c8f91686d816e8e7661013a1b8e094bb5eaec513409a12c5a46316 WatchSource:0}: Error finding container 3ddf1c06d4c8f91686d816e8e7661013a1b8e094bb5eaec513409a12c5a46316: Status 404 returned error can't find the container with id 3ddf1c06d4c8f91686d816e8e7661013a1b8e094bb5eaec513409a12c5a46316
Jul 24 23:19:07.445112 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:07.445094    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:19:07.659444 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:07.659407    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm" event={"ID":"c648bad5-f6cb-4964-ae31-dc720f31dd8c","Type":"ContainerStarted","Data":"3ddf1c06d4c8f91686d816e8e7661013a1b8e094bb5eaec513409a12c5a46316"}
Jul 24 23:19:08.665068 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:08.665031    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm" event={"ID":"c648bad5-f6cb-4964-ae31-dc720f31dd8c","Type":"ContainerStarted","Data":"b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3"}
Jul 24 23:19:08.665457 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:08.665097    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:08.682527 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:08.682482    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm" podStartSLOduration=3.036637448 podStartE2EDuration="3.682465231s" totalImagesPullingTime="645.827783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:19:08.681173063 +0000 UTC m=+1605.958206830" watchObservedRunningTime="2026-07-24 23:19:08.682465231 +0000 UTC m=+1605.959498774"
Jul 24 23:19:14.675303 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:14.675219    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:30.780897 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:30.780863    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"]
Jul 24 23:19:30.784133 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:30.784117    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"
Jul 24 23:19:30.786687 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:30.786668    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:19:30.793739 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:30.793715    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"]
Jul 24 23:19:30.918295 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:30.918263    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c9d37d-8c24-4c00-8b41-f3a54c74d52b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a599b375-qwflr\" (UID: \"a5c9d37d-8c24-4c00-8b41-f3a54c74d52b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"
Jul 24 23:19:31.019259 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:31.019228    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c9d37d-8c24-4c00-8b41-f3a54c74d52b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a599b375-qwflr\" (UID: \"a5c9d37d-8c24-4c00-8b41-f3a54c74d52b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"
Jul 24 23:19:31.019888 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:31.019870    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c9d37d-8c24-4c00-8b41-f3a54c74d52b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a599b375-qwflr\" (UID: \"a5c9d37d-8c24-4c00-8b41-f3a54c74d52b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"
Jul 24 23:19:31.095056 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:31.094988    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"
Jul 24 23:19:31.222506 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:31.222480    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"]
Jul 24 23:19:31.224134 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:19:31.224110    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c9d37d_8c24_4c00_8b41_f3a54c74d52b.slice/crio-72a33967a0a53f36a0c42a1e63b5d9d04448b18d0fa2ad19a57c10d442bf604b WatchSource:0}: Error finding container 72a33967a0a53f36a0c42a1e63b5d9d04448b18d0fa2ad19a57c10d442bf604b: Status 404 returned error can't find the container with id 72a33967a0a53f36a0c42a1e63b5d9d04448b18d0fa2ad19a57c10d442bf604b
Jul 24 23:19:31.766689 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:31.766650    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr" event={"ID":"a5c9d37d-8c24-4c00-8b41-f3a54c74d52b","Type":"ContainerStarted","Data":"72a33967a0a53f36a0c42a1e63b5d9d04448b18d0fa2ad19a57c10d442bf604b"}
Jul 24 23:19:34.781981 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:34.781955    2568 generic.go:356] "Generic (PLEG): container finished" podID="a5c9d37d-8c24-4c00-8b41-f3a54c74d52b" containerID="2d1a7bd4f8c18e82ba5f87c3115c329b87704524b0bde1746bd2e28fa69348ab" exitCode=0
Jul 24 23:19:34.782322 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:34.782041    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr" event={"ID":"a5c9d37d-8c24-4c00-8b41-f3a54c74d52b","Type":"ContainerDied","Data":"2d1a7bd4f8c18e82ba5f87c3115c329b87704524b0bde1746bd2e28fa69348ab"}
Jul 24 23:19:35.910624 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:35.910603    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"
Jul 24 23:19:36.062140 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:36.062064    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c9d37d-8c24-4c00-8b41-f3a54c74d52b-maas-api-service-ca\") pod \"a5c9d37d-8c24-4c00-8b41-f3a54c74d52b\" (UID: \"a5c9d37d-8c24-4c00-8b41-f3a54c74d52b\") "
Jul 24 23:19:36.062435 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:36.062412    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5c9d37d-8c24-4c00-8b41-f3a54c74d52b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5c9d37d-8c24-4c00-8b41-f3a54c74d52b" (UID: "a5c9d37d-8c24-4c00-8b41-f3a54c74d52b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:19:36.163697 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:36.163642    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c9d37d-8c24-4c00-8b41-f3a54c74d52b-maas-api-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:19:36.792586 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:36.792558    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"
Jul 24 23:19:36.792586 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:36.792567    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr" event={"ID":"a5c9d37d-8c24-4c00-8b41-f3a54c74d52b","Type":"ContainerDied","Data":"72a33967a0a53f36a0c42a1e63b5d9d04448b18d0fa2ad19a57c10d442bf604b"}
Jul 24 23:19:36.792838 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:36.792609    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72a33967a0a53f36a0c42a1e63b5d9d04448b18d0fa2ad19a57c10d442bf604b"
Jul 24 23:19:41.161053 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.161010    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"]
Jul 24 23:19:41.161516 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.161291    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm" podUID="c648bad5-f6cb-4964-ae31-dc720f31dd8c" containerName="maas-api" containerID="cri-o://b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3" gracePeriod=30
Jul 24 23:19:41.406021 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.405998    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:41.514428 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.514395    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls\") pod \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") "
Jul 24 23:19:41.514623 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.514438    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c648bad5-f6cb-4964-ae31-dc720f31dd8c-openshift-service-ca\") pod \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") "
Jul 24 23:19:41.514623 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.514486    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nftqc\" (UniqueName: \"kubernetes.io/projected/c648bad5-f6cb-4964-ae31-dc720f31dd8c-kube-api-access-nftqc\") pod \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\" (UID: \"c648bad5-f6cb-4964-ae31-dc720f31dd8c\") "
Jul 24 23:19:41.514864 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.514838    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c648bad5-f6cb-4964-ae31-dc720f31dd8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c648bad5-f6cb-4964-ae31-dc720f31dd8c" (UID: "c648bad5-f6cb-4964-ae31-dc720f31dd8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:19:41.516999 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.516973    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c648bad5-f6cb-4964-ae31-dc720f31dd8c-kube-api-access-nftqc" (OuterVolumeSpecName: "kube-api-access-nftqc") pod "c648bad5-f6cb-4964-ae31-dc720f31dd8c" (UID: "c648bad5-f6cb-4964-ae31-dc720f31dd8c"). InnerVolumeSpecName "kube-api-access-nftqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:19:41.517335 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.517303    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c648bad5-f6cb-4964-ae31-dc720f31dd8c" (UID: "c648bad5-f6cb-4964-ae31-dc720f31dd8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:19:41.615480 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.615447    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c648bad5-f6cb-4964-ae31-dc720f31dd8c-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:19:41.615634 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.615494    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c648bad5-f6cb-4964-ae31-dc720f31dd8c-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:19:41.615634 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.615506    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nftqc\" (UniqueName: \"kubernetes.io/projected/c648bad5-f6cb-4964-ae31-dc720f31dd8c-kube-api-access-nftqc\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:19:41.816215 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.816130    2568 generic.go:356] "Generic (PLEG): container finished" podID="c648bad5-f6cb-4964-ae31-dc720f31dd8c" containerID="b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3" exitCode=0
Jul 24 23:19:41.816215 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.816203    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"
Jul 24 23:19:41.816476 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.816207    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm" event={"ID":"c648bad5-f6cb-4964-ae31-dc720f31dd8c","Type":"ContainerDied","Data":"b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3"}
Jul 24 23:19:41.816476 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.816331    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm" event={"ID":"c648bad5-f6cb-4964-ae31-dc720f31dd8c","Type":"ContainerDied","Data":"3ddf1c06d4c8f91686d816e8e7661013a1b8e094bb5eaec513409a12c5a46316"}
Jul 24 23:19:41.816476 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.816359    2568 scope.go:117] "RemoveContainer" containerID="b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3"
Jul 24 23:19:41.825701 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.825667    2568 scope.go:117] "RemoveContainer" containerID="b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3"
Jul 24 23:19:41.826027 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:19:41.825980    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3\": container with ID starting with b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3 not found: ID does not exist" containerID="b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3"
Jul 24 23:19:41.826134 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.826037    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3"} err="failed to get container status \"b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3\": rpc error: code = NotFound desc = could not find container \"b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3\": container with ID starting with b71c1bbcc8ada54d91a90a88d3d00c626c91fe876cdbe7ee49d64bc4f82b46f3 not found: ID does not exist"
Jul 24 23:19:41.839788 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.839762    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"]
Jul 24 23:19:41.845239 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:41.845220    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a599b375-79fcd5cb9-w57rm"]
Jul 24 23:19:43.257379 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:43.257346    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c648bad5-f6cb-4964-ae31-dc720f31dd8c" path="/var/lib/kubelet/pods/c648bad5-f6cb-4964-ae31-dc720f31dd8c/volumes"
Jul 24 23:19:45.850476 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:45.850448    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"]
Jul 24 23:19:45.855289 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:45.855261    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a599b375-qwflr"]
Jul 24 23:19:47.258002 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:47.257950    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5c9d37d-8c24-4c00-8b41-f3a54c74d52b" path="/var/lib/kubelet/pods/a5c9d37d-8c24-4c00-8b41-f3a54c74d52b/volumes"
Jul 24 23:19:49.766845 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.766813    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"]
Jul 24 23:19:49.767201 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.767166    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c648bad5-f6cb-4964-ae31-dc720f31dd8c" containerName="maas-api"
Jul 24 23:19:49.767201 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.767176    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c648bad5-f6cb-4964-ae31-dc720f31dd8c" containerName="maas-api"
Jul 24 23:19:49.767201 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.767191    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5c9d37d-8c24-4c00-8b41-f3a54c74d52b" containerName="revoke-keys"
Jul 24 23:19:49.767201 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.767197    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5c9d37d-8c24-4c00-8b41-f3a54c74d52b" containerName="revoke-keys"
Jul 24 23:19:49.767352 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.767282    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c648bad5-f6cb-4964-ae31-dc720f31dd8c" containerName="maas-api"
Jul 24 23:19:49.767352 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.767291    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5c9d37d-8c24-4c00-8b41-f3a54c74d52b" containerName="revoke-keys"
Jul 24 23:19:49.774675 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.774659    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:49.777811 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.777794    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-629f57d4-serving-cert\""
Jul 24 23:19:49.783039 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.783015    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"]
Jul 24 23:19:49.790055 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.790034    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqlk5\" (UniqueName: \"kubernetes.io/projected/7e143932-d291-4d7b-a181-afac4907935f-kube-api-access-cqlk5\") pod \"maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:49.790151 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.790078    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e143932-d291-4d7b-a181-afac4907935f-openshift-service-ca\") pod \"maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:49.790229 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.790203    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e143932-d291-4d7b-a181-afac4907935f-maas-api-tls\") pod \"maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:49.891448 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.891420    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqlk5\" (UniqueName: \"kubernetes.io/projected/7e143932-d291-4d7b-a181-afac4907935f-kube-api-access-cqlk5\") pod \"maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:49.891573 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.891462    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e143932-d291-4d7b-a181-afac4907935f-openshift-service-ca\") pod \"maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:49.891573 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.891516    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e143932-d291-4d7b-a181-afac4907935f-maas-api-tls\") pod \"maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:49.891717 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:19:49.891598    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-serving-cert: secret "maas-api-e2e-migrate-629f57d4-serving-cert" not found
Jul 24 23:19:49.891717 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:19:49.891671    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e143932-d291-4d7b-a181-afac4907935f-maas-api-tls podName:7e143932-d291-4d7b-a181-afac4907935f nodeName:}" failed. No retries permitted until 2026-07-24 23:19:50.391656573 +0000 UTC m=+1647.668690095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7e143932-d291-4d7b-a181-afac4907935f-maas-api-tls") pod "maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2" (UID: "7e143932-d291-4d7b-a181-afac4907935f") : secret "maas-api-e2e-migrate-629f57d4-serving-cert" not found
Jul 24 23:19:49.892184 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.892161    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e143932-d291-4d7b-a181-afac4907935f-openshift-service-ca\") pod \"maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:49.900587 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:49.900566    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqlk5\" (UniqueName: \"kubernetes.io/projected/7e143932-d291-4d7b-a181-afac4907935f-kube-api-access-cqlk5\") pod \"maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:50.396182 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.396150    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e143932-d291-4d7b-a181-afac4907935f-maas-api-tls\") pod \"maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:50.398455 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.398428    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e143932-d291-4d7b-a181-afac4907935f-maas-api-tls\") pod \"maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:50.419202 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.419172    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"]
Jul 24 23:19:50.423821 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.423801    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:19:50.426469 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.426446    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-629f57d4\""
Jul 24 23:19:50.426562 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.426530    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-629f57d4-dockercfg-c4tqw\""
Jul 24 23:19:50.431660 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.431626    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"]
Jul 24 23:19:50.497591 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.497566    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m682z\" (UniqueName: \"kubernetes.io/projected/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-kube-api-access-m682z\") pod \"payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx\" (UID: \"67c7af8a-6ad0-404f-ab0e-0caf1000af5e\") " pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:19:50.497711 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.497599    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-plugins-config-volume\") pod \"payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx\" (UID: \"67c7af8a-6ad0-404f-ab0e-0caf1000af5e\") " pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:19:50.598878 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.598848    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m682z\" (UniqueName: \"kubernetes.io/projected/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-kube-api-access-m682z\") pod \"payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx\" (UID: \"67c7af8a-6ad0-404f-ab0e-0caf1000af5e\") " pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:19:50.598878 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.598881    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-plugins-config-volume\") pod \"payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx\" (UID: \"67c7af8a-6ad0-404f-ab0e-0caf1000af5e\") " pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:19:50.599562 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.599544    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-plugins-config-volume\") pod \"payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx\" (UID: \"67c7af8a-6ad0-404f-ab0e-0caf1000af5e\") " pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:19:50.607226 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.607204    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m682z\" (UniqueName: \"kubernetes.io/projected/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-kube-api-access-m682z\") pod \"payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx\" (UID: \"67c7af8a-6ad0-404f-ab0e-0caf1000af5e\") " pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:19:50.685846 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.685788    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:50.736453 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.736419    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:19:50.820158 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.820128    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"]
Jul 24 23:19:50.823464 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:19:50.823437    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e143932_d291_4d7b_a181_afac4907935f.slice/crio-3f1447d5bf491141f7c1f7d29695591b746be440b0483c3e435bae87ece723f6 WatchSource:0}: Error finding container 3f1447d5bf491141f7c1f7d29695591b746be440b0483c3e435bae87ece723f6: Status 404 returned error can't find the container with id 3f1447d5bf491141f7c1f7d29695591b746be440b0483c3e435bae87ece723f6
Jul 24 23:19:50.861457 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.861409    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2" event={"ID":"7e143932-d291-4d7b-a181-afac4907935f","Type":"ContainerStarted","Data":"3f1447d5bf491141f7c1f7d29695591b746be440b0483c3e435bae87ece723f6"}
Jul 24 23:19:50.874351 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:50.874328    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"]
Jul 24 23:19:50.876020 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:19:50.875991    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67c7af8a_6ad0_404f_ab0e_0caf1000af5e.slice/crio-95156a958f687e5ea950f99c3ecc1fe8220d91b9a8addbea8f47500f4b9bbcd0 WatchSource:0}: Error finding container 95156a958f687e5ea950f99c3ecc1fe8220d91b9a8addbea8f47500f4b9bbcd0: Status 404 returned error can't find the container with id 95156a958f687e5ea950f99c3ecc1fe8220d91b9a8addbea8f47500f4b9bbcd0
Jul 24 23:19:51.868192 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:51.868143    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx" event={"ID":"67c7af8a-6ad0-404f-ab0e-0caf1000af5e","Type":"ContainerStarted","Data":"95156a958f687e5ea950f99c3ecc1fe8220d91b9a8addbea8f47500f4b9bbcd0"}
Jul 24 23:19:51.870032 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:51.870003    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2" event={"ID":"7e143932-d291-4d7b-a181-afac4907935f","Type":"ContainerStarted","Data":"01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27"}
Jul 24 23:19:51.870181 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:51.870116    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:19:51.887356 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:51.887314    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2" podStartSLOduration=2.5068460679999998 podStartE2EDuration="2.887299216s" totalImagesPullingTime="380.453148ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:19:51.885572898 +0000 UTC m=+1649.162606442" watchObservedRunningTime="2026-07-24 23:19:51.887299216 +0000 UTC m=+1649.164332763"
Jul 24 23:19:53.881212 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:53.881179    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx" event={"ID":"67c7af8a-6ad0-404f-ab0e-0caf1000af5e","Type":"ContainerStarted","Data":"02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b"}
Jul 24 23:19:53.881649 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:53.881315    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:19:53.896975 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:53.896930    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx" podStartSLOduration=1.66640755 podStartE2EDuration="3.896915928s" totalImagesPullingTime="2.230508378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:19:53.89550789 +0000 UTC m=+1651.172541463" watchObservedRunningTime="2026-07-24 23:19:53.896915928 +0000 UTC m=+1651.173949473"
Jul 24 23:19:57.880602 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:19:57.880576    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:20:04.887505 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:04.887475    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:20:24.921131 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:24.921099    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"]
Jul 24 23:20:24.921589 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:24.921330    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2" podUID="7e143932-d291-4d7b-a181-afac4907935f" containerName="maas-api" containerID="cri-o://01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27" gracePeriod=30
Jul 24 23:20:24.976905 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:24.976875    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"]
Jul 24 23:20:24.977188 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:24.977131    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx" podUID="67c7af8a-6ad0-404f-ab0e-0caf1000af5e" containerName="payload-processing" containerID="cri-o://02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b" gracePeriod=30
Jul 24 23:20:25.216941 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.216914    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:20:25.245994 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.245974    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:20:25.327007 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.326977    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqlk5\" (UniqueName: \"kubernetes.io/projected/7e143932-d291-4d7b-a181-afac4907935f-kube-api-access-cqlk5\") pod \"7e143932-d291-4d7b-a181-afac4907935f\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") "
Jul 24 23:20:25.327177 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.327069    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e143932-d291-4d7b-a181-afac4907935f-maas-api-tls\") pod \"7e143932-d291-4d7b-a181-afac4907935f\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") "
Jul 24 23:20:25.327177 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.327112    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e143932-d291-4d7b-a181-afac4907935f-openshift-service-ca\") pod \"7e143932-d291-4d7b-a181-afac4907935f\" (UID: \"7e143932-d291-4d7b-a181-afac4907935f\") "
Jul 24 23:20:25.327556 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.327528    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e143932-d291-4d7b-a181-afac4907935f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7e143932-d291-4d7b-a181-afac4907935f" (UID: "7e143932-d291-4d7b-a181-afac4907935f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:20:25.329095 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.329068    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e143932-d291-4d7b-a181-afac4907935f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7e143932-d291-4d7b-a181-afac4907935f" (UID: "7e143932-d291-4d7b-a181-afac4907935f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:20:25.329158 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.329091    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e143932-d291-4d7b-a181-afac4907935f-kube-api-access-cqlk5" (OuterVolumeSpecName: "kube-api-access-cqlk5") pod "7e143932-d291-4d7b-a181-afac4907935f" (UID: "7e143932-d291-4d7b-a181-afac4907935f"). InnerVolumeSpecName "kube-api-access-cqlk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:20:25.428831 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.428741    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m682z\" (UniqueName: \"kubernetes.io/projected/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-kube-api-access-m682z\") pod \"67c7af8a-6ad0-404f-ab0e-0caf1000af5e\" (UID: \"67c7af8a-6ad0-404f-ab0e-0caf1000af5e\") "
Jul 24 23:20:25.428831 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.428794    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-plugins-config-volume\") pod \"67c7af8a-6ad0-404f-ab0e-0caf1000af5e\" (UID: \"67c7af8a-6ad0-404f-ab0e-0caf1000af5e\") "
Jul 24 23:20:25.429048 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.429009    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqlk5\" (UniqueName: \"kubernetes.io/projected/7e143932-d291-4d7b-a181-afac4907935f-kube-api-access-cqlk5\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:20:25.429048 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.429022    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e143932-d291-4d7b-a181-afac4907935f-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:20:25.429048 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.429031    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e143932-d291-4d7b-a181-afac4907935f-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:20:25.429206 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.429131    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "67c7af8a-6ad0-404f-ab0e-0caf1000af5e" (UID: "67c7af8a-6ad0-404f-ab0e-0caf1000af5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:20:25.430864 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.430848    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-kube-api-access-m682z" (OuterVolumeSpecName: "kube-api-access-m682z") pod "67c7af8a-6ad0-404f-ab0e-0caf1000af5e" (UID: "67c7af8a-6ad0-404f-ab0e-0caf1000af5e"). InnerVolumeSpecName "kube-api-access-m682z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:20:25.529989 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.529963    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m682z\" (UniqueName: \"kubernetes.io/projected/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-kube-api-access-m682z\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:20:25.529989 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:25.529985    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67c7af8a-6ad0-404f-ab0e-0caf1000af5e-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:20:26.036155 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.036123    2568 generic.go:356] "Generic (PLEG): container finished" podID="67c7af8a-6ad0-404f-ab0e-0caf1000af5e" containerID="02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b" exitCode=0
Jul 24 23:20:26.036694 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.036162    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"
Jul 24 23:20:26.036694 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.036210    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx" event={"ID":"67c7af8a-6ad0-404f-ab0e-0caf1000af5e","Type":"ContainerDied","Data":"02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b"}
Jul 24 23:20:26.036694 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.036275    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx" event={"ID":"67c7af8a-6ad0-404f-ab0e-0caf1000af5e","Type":"ContainerDied","Data":"95156a958f687e5ea950f99c3ecc1fe8220d91b9a8addbea8f47500f4b9bbcd0"}
Jul 24 23:20:26.036694 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.036303    2568 scope.go:117] "RemoveContainer" containerID="02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b"
Jul 24 23:20:26.037757 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.037534    2568 generic.go:356] "Generic (PLEG): container finished" podID="7e143932-d291-4d7b-a181-afac4907935f" containerID="01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27" exitCode=0
Jul 24 23:20:26.037757 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.037575    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2" event={"ID":"7e143932-d291-4d7b-a181-afac4907935f","Type":"ContainerDied","Data":"01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27"}
Jul 24 23:20:26.037757 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.037586    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"
Jul 24 23:20:26.037757 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.037625    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2" event={"ID":"7e143932-d291-4d7b-a181-afac4907935f","Type":"ContainerDied","Data":"3f1447d5bf491141f7c1f7d29695591b746be440b0483c3e435bae87ece723f6"}
Jul 24 23:20:26.051259 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.051223    2568 scope.go:117] "RemoveContainer" containerID="02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b"
Jul 24 23:20:26.052161 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:20:26.052133    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b\": container with ID starting with 02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b not found: ID does not exist" containerID="02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b"
Jul 24 23:20:26.052285 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.052172    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b"} err="failed to get container status \"02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b\": rpc error: code = NotFound desc = could not find container \"02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b\": container with ID starting with 02b85d8e42955111ad9335e731151ef8ff11ed1bc67b5e1387e3f51e8979436b not found: ID does not exist"
Jul 24 23:20:26.052285 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.052196    2568 scope.go:117] "RemoveContainer" containerID="01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27"
Jul 24 23:20:26.060505 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.060491    2568 scope.go:117] "RemoveContainer" containerID="01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27"
Jul 24 23:20:26.060751 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:20:26.060733    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27\": container with ID starting with 01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27 not found: ID does not exist" containerID="01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27"
Jul 24 23:20:26.060803 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.060758    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27"} err="failed to get container status \"01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27\": rpc error: code = NotFound desc = could not find container \"01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27\": container with ID starting with 01f84fd7b70314aa34269d0710aab8da210278d3e99b9ff22dfc48d1548bbd27 not found: ID does not exist"
Jul 24 23:20:26.066806 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.066782    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"]
Jul 24 23:20:26.070114 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.070094    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-629f57d4-54b78749fd-m6gt2"]
Jul 24 23:20:26.080499 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.080476    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"]
Jul 24 23:20:26.084166 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:26.084148    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-629f57d4-67c598d465-7pkmx"]
Jul 24 23:20:27.258236 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:27.258207    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67c7af8a-6ad0-404f-ab0e-0caf1000af5e" path="/var/lib/kubelet/pods/67c7af8a-6ad0-404f-ab0e-0caf1000af5e/volumes"
Jul 24 23:20:27.258625 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:27.258606    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e143932-d291-4d7b-a181-afac4907935f" path="/var/lib/kubelet/pods/7e143932-d291-4d7b-a181-afac4907935f/volumes"
Jul 24 23:20:32.807791 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.807762    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"]
Jul 24 23:20:32.808188 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.808130    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67c7af8a-6ad0-404f-ab0e-0caf1000af5e" containerName="payload-processing"
Jul 24 23:20:32.808188 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.808142    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="67c7af8a-6ad0-404f-ab0e-0caf1000af5e" containerName="payload-processing"
Jul 24 23:20:32.808188 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.808158    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e143932-d291-4d7b-a181-afac4907935f" containerName="maas-api"
Jul 24 23:20:32.808188 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.808163    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e143932-d291-4d7b-a181-afac4907935f" containerName="maas-api"
Jul 24 23:20:32.808341 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.808226    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e143932-d291-4d7b-a181-afac4907935f" containerName="maas-api"
Jul 24 23:20:32.808341 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.808236    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="67c7af8a-6ad0-404f-ab0e-0caf1000af5e" containerName="payload-processing"
Jul 24 23:20:32.812858 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.812842    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:32.815919 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.815898    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2af5eeed-serving-cert\""
Jul 24 23:20:32.821928 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.821903    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"]
Jul 24 23:20:32.897226 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.897191    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwms5\" (UniqueName: \"kubernetes.io/projected/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-kube-api-access-vwms5\") pod \"maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:32.897377 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.897302    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-openshift-service-ca\") pod \"maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:32.897377 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.897339    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-maas-api-tls\") pod \"maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:32.998192 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.998159    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-openshift-service-ca\") pod \"maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:32.998376 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.998199    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-maas-api-tls\") pod \"maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:32.998376 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.998244    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwms5\" (UniqueName: \"kubernetes.io/projected/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-kube-api-access-vwms5\") pod \"maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:32.998868 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:32.998840    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-openshift-service-ca\") pod \"maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:33.000697 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.000673    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-maas-api-tls\") pod \"maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:33.007670 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.007640    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwms5\" (UniqueName: \"kubernetes.io/projected/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-kube-api-access-vwms5\") pod \"maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:33.123662 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.123576    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:33.455853 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.455829    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"]
Jul 24 23:20:33.457524 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:20:33.457500    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52321f9a_9b97_40fe_8bc8_4cb4f62804c7.slice/crio-7e47fcb186b0778dfbff088619bc4439e5aac3467bb28443e0e265e6aeb0a24d WatchSource:0}: Error finding container 7e47fcb186b0778dfbff088619bc4439e5aac3467bb28443e0e265e6aeb0a24d: Status 404 returned error can't find the container with id 7e47fcb186b0778dfbff088619bc4439e5aac3467bb28443e0e265e6aeb0a24d
Jul 24 23:20:33.535874 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.535843    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"]
Jul 24 23:20:33.540558 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.540542    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:20:33.543142 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.543125    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2af5eeed\""
Jul 24 23:20:33.543214 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.543127    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-2af5eeed-dockercfg-dqnv5\""
Jul 24 23:20:33.549117 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.549096    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"]
Jul 24 23:20:33.576715 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.576690    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"]
Jul 24 23:20:33.580604 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.580582    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:20:33.589286 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.589263    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"]
Jul 24 23:20:33.603717 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.603669    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4e91ba5-c5ff-4dda-9156-18a966b35942-plugins-config-volume\") pod \"payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj\" (UID: \"b4e91ba5-c5ff-4dda-9156-18a966b35942\") " pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:20:33.603822 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.603733    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlmgm\" (UniqueName: \"kubernetes.io/projected/0accc2b2-d42b-4c54-92a3-e6c45feba62c-kube-api-access-qlmgm\") pod \"payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g\" (UID: \"0accc2b2-d42b-4c54-92a3-e6c45feba62c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:20:33.603822 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.603768    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5wt2\" (UniqueName: \"kubernetes.io/projected/b4e91ba5-c5ff-4dda-9156-18a966b35942-kube-api-access-h5wt2\") pod \"payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj\" (UID: \"b4e91ba5-c5ff-4dda-9156-18a966b35942\") " pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:20:33.603907 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.603870    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0accc2b2-d42b-4c54-92a3-e6c45feba62c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g\" (UID: \"0accc2b2-d42b-4c54-92a3-e6c45feba62c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:20:33.705148 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.705070    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5wt2\" (UniqueName: \"kubernetes.io/projected/b4e91ba5-c5ff-4dda-9156-18a966b35942-kube-api-access-h5wt2\") pod \"payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj\" (UID: \"b4e91ba5-c5ff-4dda-9156-18a966b35942\") " pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:20:33.705321 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.705154    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0accc2b2-d42b-4c54-92a3-e6c45feba62c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g\" (UID: \"0accc2b2-d42b-4c54-92a3-e6c45feba62c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:20:33.705321 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.705178    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4e91ba5-c5ff-4dda-9156-18a966b35942-plugins-config-volume\") pod \"payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj\" (UID: \"b4e91ba5-c5ff-4dda-9156-18a966b35942\") " pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:20:33.705321 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.705206    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlmgm\" (UniqueName: \"kubernetes.io/projected/0accc2b2-d42b-4c54-92a3-e6c45feba62c-kube-api-access-qlmgm\") pod \"payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g\" (UID: \"0accc2b2-d42b-4c54-92a3-e6c45feba62c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:20:33.705868 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.705846    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0accc2b2-d42b-4c54-92a3-e6c45feba62c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g\" (UID: \"0accc2b2-d42b-4c54-92a3-e6c45feba62c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:20:33.705941 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.705874    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4e91ba5-c5ff-4dda-9156-18a966b35942-plugins-config-volume\") pod \"payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj\" (UID: \"b4e91ba5-c5ff-4dda-9156-18a966b35942\") " pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:20:33.713182 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.713153    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlmgm\" (UniqueName: \"kubernetes.io/projected/0accc2b2-d42b-4c54-92a3-e6c45feba62c-kube-api-access-qlmgm\") pod \"payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g\" (UID: \"0accc2b2-d42b-4c54-92a3-e6c45feba62c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:20:33.713294 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.713215    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5wt2\" (UniqueName: \"kubernetes.io/projected/b4e91ba5-c5ff-4dda-9156-18a966b35942-kube-api-access-h5wt2\") pod \"payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj\" (UID: \"b4e91ba5-c5ff-4dda-9156-18a966b35942\") " pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:20:33.852411 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.852384    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:20:33.892277 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:33.892236    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:20:34.005262 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:34.005227    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"]
Jul 24 23:20:34.007038 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:20:34.007008    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e91ba5_c5ff_4dda_9156_18a966b35942.slice/crio-7c78c82a16b867ac8b618376f85ec0aea5e64a77c69283dc8091d3488a07e521 WatchSource:0}: Error finding container 7c78c82a16b867ac8b618376f85ec0aea5e64a77c69283dc8091d3488a07e521: Status 404 returned error can't find the container with id 7c78c82a16b867ac8b618376f85ec0aea5e64a77c69283dc8091d3488a07e521
Jul 24 23:20:34.049012 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:34.048982    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"]
Jul 24 23:20:34.051604 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:20:34.051579    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0accc2b2_d42b_4c54_92a3_e6c45feba62c.slice/crio-27e136a82f10268eae63a4e59aae9027944bafe478e33e3c78007396a39dc61d WatchSource:0}: Error finding container 27e136a82f10268eae63a4e59aae9027944bafe478e33e3c78007396a39dc61d: Status 404 returned error can't find the container with id 27e136a82f10268eae63a4e59aae9027944bafe478e33e3c78007396a39dc61d
Jul 24 23:20:34.079402 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:34.079370    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g" event={"ID":"0accc2b2-d42b-4c54-92a3-e6c45feba62c","Type":"ContainerStarted","Data":"27e136a82f10268eae63a4e59aae9027944bafe478e33e3c78007396a39dc61d"}
Jul 24 23:20:34.081423 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:34.081395    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj" event={"ID":"b4e91ba5-c5ff-4dda-9156-18a966b35942","Type":"ContainerStarted","Data":"7c78c82a16b867ac8b618376f85ec0aea5e64a77c69283dc8091d3488a07e521"}
Jul 24 23:20:34.083053 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:34.083028    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc" event={"ID":"52321f9a-9b97-40fe-8bc8-4cb4f62804c7","Type":"ContainerStarted","Data":"77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e"}
Jul 24 23:20:34.083142 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:34.083075    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc" event={"ID":"52321f9a-9b97-40fe-8bc8-4cb4f62804c7","Type":"ContainerStarted","Data":"7e47fcb186b0778dfbff088619bc4439e5aac3467bb28443e0e265e6aeb0a24d"}
Jul 24 23:20:34.083291 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:34.083169    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:34.100198 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:34.100149    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc" podStartSLOduration=1.6917885099999999 podStartE2EDuration="2.100128429s" totalImagesPullingTime="408.339919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:20:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:20:34.09857873 +0000 UTC m=+1691.375612275" watchObservedRunningTime="2026-07-24 23:20:34.100128429 +0000 UTC m=+1691.377161977"
Jul 24 23:20:35.090012 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:35.089982    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g" event={"ID":"0accc2b2-d42b-4c54-92a3-e6c45feba62c","Type":"ContainerStarted","Data":"d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba"}
Jul 24 23:20:35.090456 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:35.090078    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:20:35.091545 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:35.091523    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj" event={"ID":"b4e91ba5-c5ff-4dda-9156-18a966b35942","Type":"ContainerStarted","Data":"9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975"}
Jul 24 23:20:35.091737 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:35.091720    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:20:35.112267 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:35.106176    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g" podStartSLOduration=2.106149876 podStartE2EDuration="2.106149876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:20:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:20:35.104983759 +0000 UTC m=+1692.382017326" watchObservedRunningTime="2026-07-24 23:20:35.106149876 +0000 UTC m=+1692.383183420"
Jul 24 23:20:35.127432 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:35.127389    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj" podStartSLOduration=2.127376692 podStartE2EDuration="2.127376692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:20:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:20:35.126536158 +0000 UTC m=+1692.403569739" watchObservedRunningTime="2026-07-24 23:20:35.127376692 +0000 UTC m=+1692.404410234"
Jul 24 23:20:40.096707 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:40.096677    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:20:46.098301 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:46.098208    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:20:46.098788 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:20:46.098304    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:21:09.328594 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.328558    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"]
Jul 24 23:21:09.329114 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.328756    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc" podUID="52321f9a-9b97-40fe-8bc8-4cb4f62804c7" containerName="maas-api" containerID="cri-o://77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e" gracePeriod=30
Jul 24 23:21:09.375740 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.373655    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"]
Jul 24 23:21:09.375740 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.373908    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj" podUID="b4e91ba5-c5ff-4dda-9156-18a966b35942" containerName="payload-processing" containerID="cri-o://9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975" gracePeriod=30
Jul 24 23:21:09.381289 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.381131    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"]
Jul 24 23:21:09.381433 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.381376    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g" podUID="0accc2b2-d42b-4c54-92a3-e6c45feba62c" containerName="payload-pre-processing" containerID="cri-o://d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba" gracePeriod=30
Jul 24 23:21:09.633291 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.633264    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:21:09.654583 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.654556    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:21:09.667708 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.667667    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:21:09.735843 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.735811    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-openshift-service-ca\") pod \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") "
Jul 24 23:21:09.736011 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.735856    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4e91ba5-c5ff-4dda-9156-18a966b35942-plugins-config-volume\") pod \"b4e91ba5-c5ff-4dda-9156-18a966b35942\" (UID: \"b4e91ba5-c5ff-4dda-9156-18a966b35942\") "
Jul 24 23:21:09.736011 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.735880    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-maas-api-tls\") pod \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") "
Jul 24 23:21:09.736011 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.735900    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0accc2b2-d42b-4c54-92a3-e6c45feba62c-plugins-config-volume\") pod \"0accc2b2-d42b-4c54-92a3-e6c45feba62c\" (UID: \"0accc2b2-d42b-4c54-92a3-e6c45feba62c\") "
Jul 24 23:21:09.736011 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.735949    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwms5\" (UniqueName: \"kubernetes.io/projected/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-kube-api-access-vwms5\") pod \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\" (UID: \"52321f9a-9b97-40fe-8bc8-4cb4f62804c7\") "
Jul 24 23:21:09.736011 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.735972    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlmgm\" (UniqueName: \"kubernetes.io/projected/0accc2b2-d42b-4c54-92a3-e6c45feba62c-kube-api-access-qlmgm\") pod \"0accc2b2-d42b-4c54-92a3-e6c45feba62c\" (UID: \"0accc2b2-d42b-4c54-92a3-e6c45feba62c\") "
Jul 24 23:21:09.736329 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.736066    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5wt2\" (UniqueName: \"kubernetes.io/projected/b4e91ba5-c5ff-4dda-9156-18a966b35942-kube-api-access-h5wt2\") pod \"b4e91ba5-c5ff-4dda-9156-18a966b35942\" (UID: \"b4e91ba5-c5ff-4dda-9156-18a966b35942\") "
Jul 24 23:21:09.736388 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.736367    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0accc2b2-d42b-4c54-92a3-e6c45feba62c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0accc2b2-d42b-4c54-92a3-e6c45feba62c" (UID: "0accc2b2-d42b-4c54-92a3-e6c45feba62c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:21:09.736443 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.736386    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "52321f9a-9b97-40fe-8bc8-4cb4f62804c7" (UID: "52321f9a-9b97-40fe-8bc8-4cb4f62804c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:21:09.736443 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.736391    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4e91ba5-c5ff-4dda-9156-18a966b35942-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b4e91ba5-c5ff-4dda-9156-18a966b35942" (UID: "b4e91ba5-c5ff-4dda-9156-18a966b35942"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:21:09.737959 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.737934    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "52321f9a-9b97-40fe-8bc8-4cb4f62804c7" (UID: "52321f9a-9b97-40fe-8bc8-4cb4f62804c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:21:09.738146 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.738129    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e91ba5-c5ff-4dda-9156-18a966b35942-kube-api-access-h5wt2" (OuterVolumeSpecName: "kube-api-access-h5wt2") pod "b4e91ba5-c5ff-4dda-9156-18a966b35942" (UID: "b4e91ba5-c5ff-4dda-9156-18a966b35942"). InnerVolumeSpecName "kube-api-access-h5wt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:21:09.738441 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.738417    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-kube-api-access-vwms5" (OuterVolumeSpecName: "kube-api-access-vwms5") pod "52321f9a-9b97-40fe-8bc8-4cb4f62804c7" (UID: "52321f9a-9b97-40fe-8bc8-4cb4f62804c7"). InnerVolumeSpecName "kube-api-access-vwms5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:21:09.738441 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.738424    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0accc2b2-d42b-4c54-92a3-e6c45feba62c-kube-api-access-qlmgm" (OuterVolumeSpecName: "kube-api-access-qlmgm") pod "0accc2b2-d42b-4c54-92a3-e6c45feba62c" (UID: "0accc2b2-d42b-4c54-92a3-e6c45feba62c"). InnerVolumeSpecName "kube-api-access-qlmgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:21:09.837642 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.837610    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4e91ba5-c5ff-4dda-9156-18a966b35942-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:09.837642 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.837638    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:09.837642 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.837647    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0accc2b2-d42b-4c54-92a3-e6c45feba62c-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:09.837864 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.837657    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwms5\" (UniqueName: \"kubernetes.io/projected/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-kube-api-access-vwms5\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:09.837864 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.837665    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlmgm\" (UniqueName: \"kubernetes.io/projected/0accc2b2-d42b-4c54-92a3-e6c45feba62c-kube-api-access-qlmgm\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:09.837864 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.837674    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5wt2\" (UniqueName: \"kubernetes.io/projected/b4e91ba5-c5ff-4dda-9156-18a966b35942-kube-api-access-h5wt2\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:09.837864 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:09.837682    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52321f9a-9b97-40fe-8bc8-4cb4f62804c7-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:10.271292 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.271261    2568 generic.go:356] "Generic (PLEG): container finished" podID="0accc2b2-d42b-4c54-92a3-e6c45feba62c" containerID="d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba" exitCode=0
Jul 24 23:21:10.271472 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.271302    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"
Jul 24 23:21:10.271472 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.271340    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g" event={"ID":"0accc2b2-d42b-4c54-92a3-e6c45feba62c","Type":"ContainerDied","Data":"d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba"}
Jul 24 23:21:10.271472 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.271393    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g" event={"ID":"0accc2b2-d42b-4c54-92a3-e6c45feba62c","Type":"ContainerDied","Data":"27e136a82f10268eae63a4e59aae9027944bafe478e33e3c78007396a39dc61d"}
Jul 24 23:21:10.271472 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.271424    2568 scope.go:117] "RemoveContainer" containerID="d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba"
Jul 24 23:21:10.272770 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.272743    2568 generic.go:356] "Generic (PLEG): container finished" podID="b4e91ba5-c5ff-4dda-9156-18a966b35942" containerID="9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975" exitCode=0
Jul 24 23:21:10.272889 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.272792    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"
Jul 24 23:21:10.272889 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.272869    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj" event={"ID":"b4e91ba5-c5ff-4dda-9156-18a966b35942","Type":"ContainerDied","Data":"9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975"}
Jul 24 23:21:10.273021 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.272928    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj" event={"ID":"b4e91ba5-c5ff-4dda-9156-18a966b35942","Type":"ContainerDied","Data":"7c78c82a16b867ac8b618376f85ec0aea5e64a77c69283dc8091d3488a07e521"}
Jul 24 23:21:10.274128 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.274107    2568 generic.go:356] "Generic (PLEG): container finished" podID="52321f9a-9b97-40fe-8bc8-4cb4f62804c7" containerID="77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e" exitCode=0
Jul 24 23:21:10.274224 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.274146    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"
Jul 24 23:21:10.274224 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.274159    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc" event={"ID":"52321f9a-9b97-40fe-8bc8-4cb4f62804c7","Type":"ContainerDied","Data":"77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e"}
Jul 24 23:21:10.274224 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.274195    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc" event={"ID":"52321f9a-9b97-40fe-8bc8-4cb4f62804c7","Type":"ContainerDied","Data":"7e47fcb186b0778dfbff088619bc4439e5aac3467bb28443e0e265e6aeb0a24d"}
Jul 24 23:21:10.280409 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.280364    2568 scope.go:117] "RemoveContainer" containerID="d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba"
Jul 24 23:21:10.280693 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:21:10.280651    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba\": container with ID starting with d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba not found: ID does not exist" containerID="d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba"
Jul 24 23:21:10.280804 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.280696    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba"} err="failed to get container status \"d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba\": rpc error: code = NotFound desc = could not find container \"d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba\": container with ID starting with d47da0a6fa8ffaca4fdfd6b2db4de21c4fd35d5659c59df8b6878b79cc9287ba not found: ID does not exist"
Jul 24 23:21:10.280804 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.280712    2568 scope.go:117] "RemoveContainer" containerID="9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975"
Jul 24 23:21:10.289653 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.289635    2568 scope.go:117] "RemoveContainer" containerID="9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975"
Jul 24 23:21:10.289896 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:21:10.289879    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975\": container with ID starting with 9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975 not found: ID does not exist" containerID="9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975"
Jul 24 23:21:10.289939 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.289903    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975"} err="failed to get container status \"9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975\": rpc error: code = NotFound desc = could not find container \"9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975\": container with ID starting with 9c010c5da5d0833875c5ef3b62e71f4d16535ddffc3454c169ced384679ef975 not found: ID does not exist"
Jul 24 23:21:10.289939 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.289919    2568 scope.go:117] "RemoveContainer" containerID="77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e"
Jul 24 23:21:10.297197 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.297173    2568 scope.go:117] "RemoveContainer" containerID="77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e"
Jul 24 23:21:10.297616 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:21:10.297477    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e\": container with ID starting with 77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e not found: ID does not exist" containerID="77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e"
Jul 24 23:21:10.297616 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.297507    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e"} err="failed to get container status \"77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e\": rpc error: code = NotFound desc = could not find container \"77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e\": container with ID starting with 77851e2e5c254cf3afc2346798f55e815e19c11d8dcc70bbe3381a95aa3d1b3e not found: ID does not exist"
Jul 24 23:21:10.299262 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.299229    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"]
Jul 24 23:21:10.304035 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.304014    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2af5eeed-759f6d5fd9-p5zdc"]
Jul 24 23:21:10.314548 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.314525    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"]
Jul 24 23:21:10.318160 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.318140    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2af5eeed-65bc9cf4d4-99r6g"]
Jul 24 23:21:10.328584 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.328563    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"]
Jul 24 23:21:10.332109 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:10.332089    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2af5eeed-86d954c6cd-jqcjj"]
Jul 24 23:21:11.257185 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:11.257156    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0accc2b2-d42b-4c54-92a3-e6c45feba62c" path="/var/lib/kubelet/pods/0accc2b2-d42b-4c54-92a3-e6c45feba62c/volumes"
Jul 24 23:21:11.257549 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:11.257535    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52321f9a-9b97-40fe-8bc8-4cb4f62804c7" path="/var/lib/kubelet/pods/52321f9a-9b97-40fe-8bc8-4cb4f62804c7/volumes"
Jul 24 23:21:11.257837 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:11.257825    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4e91ba5-c5ff-4dda-9156-18a966b35942" path="/var/lib/kubelet/pods/b4e91ba5-c5ff-4dda-9156-18a966b35942/volumes"
Jul 24 23:21:17.902687 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.902637    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"]
Jul 24 23:21:17.905163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.903181    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e91ba5-c5ff-4dda-9156-18a966b35942" containerName="payload-processing"
Jul 24 23:21:17.905163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.903200    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e91ba5-c5ff-4dda-9156-18a966b35942" containerName="payload-processing"
Jul 24 23:21:17.905163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.903229    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52321f9a-9b97-40fe-8bc8-4cb4f62804c7" containerName="maas-api"
Jul 24 23:21:17.905163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.903238    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="52321f9a-9b97-40fe-8bc8-4cb4f62804c7" containerName="maas-api"
Jul 24 23:21:17.905163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.903282    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0accc2b2-d42b-4c54-92a3-e6c45feba62c" containerName="payload-pre-processing"
Jul 24 23:21:17.905163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.903292    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0accc2b2-d42b-4c54-92a3-e6c45feba62c" containerName="payload-pre-processing"
Jul 24 23:21:17.905163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.903395    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4e91ba5-c5ff-4dda-9156-18a966b35942" containerName="payload-processing"
Jul 24 23:21:17.905163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.903405    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0accc2b2-d42b-4c54-92a3-e6c45feba62c" containerName="payload-pre-processing"
Jul 24 23:21:17.905163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.903413    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="52321f9a-9b97-40fe-8bc8-4cb4f62804c7" containerName="maas-api"
Jul 24 23:21:17.906377 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.906359    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:17.909619 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.909601    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a4bebff3-serving-cert\""
Jul 24 23:21:17.917099 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:17.917077    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"]
Jul 24 23:21:18.012725 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.012684    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s49vv\" (UniqueName: \"kubernetes.io/projected/b0183855-e59a-415f-92a7-31ba78086aa1-kube-api-access-s49vv\") pod \"maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:18.012725 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.012729    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0183855-e59a-415f-92a7-31ba78086aa1-openshift-service-ca\") pod \"maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:18.012956 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.012760    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0183855-e59a-415f-92a7-31ba78086aa1-maas-api-tls\") pod \"maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:18.113664 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.113625    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s49vv\" (UniqueName: \"kubernetes.io/projected/b0183855-e59a-415f-92a7-31ba78086aa1-kube-api-access-s49vv\") pod \"maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:18.113664 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.113671    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0183855-e59a-415f-92a7-31ba78086aa1-openshift-service-ca\") pod \"maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:18.113860 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.113689    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0183855-e59a-415f-92a7-31ba78086aa1-maas-api-tls\") pod \"maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:18.114353 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.114328    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0183855-e59a-415f-92a7-31ba78086aa1-openshift-service-ca\") pod \"maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:18.116010 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.115990    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0183855-e59a-415f-92a7-31ba78086aa1-maas-api-tls\") pod \"maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:18.125053 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.125030    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s49vv\" (UniqueName: \"kubernetes.io/projected/b0183855-e59a-415f-92a7-31ba78086aa1-kube-api-access-s49vv\") pod \"maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:18.217035 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.217006    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:18.347236 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.347215    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"]
Jul 24 23:21:18.348917 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:21:18.348880    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0183855_e59a_415f_92a7_31ba78086aa1.slice/crio-f812fe191a8f515fdc772b31fc8451c3b9a99db2fa4a77b4ff08af2d3ccdf2a5 WatchSource:0}: Error finding container f812fe191a8f515fdc772b31fc8451c3b9a99db2fa4a77b4ff08af2d3ccdf2a5: Status 404 returned error can't find the container with id f812fe191a8f515fdc772b31fc8451c3b9a99db2fa4a77b4ff08af2d3ccdf2a5
Jul 24 23:21:18.588542 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.588454    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"]
Jul 24 23:21:18.593032 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.593010    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:18.595753 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.595725    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-a4bebff3\""
Jul 24 23:21:18.595753 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.595745    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-a4bebff3-dockercfg-4wgmg\""
Jul 24 23:21:18.601036 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.600980    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"]
Jul 24 23:21:18.634071 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.634034    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"]
Jul 24 23:21:18.637592 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.637573    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:18.648857 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.648835    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"]
Jul 24 23:21:18.719453 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.719421    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/220fcd47-c1db-4446-b47e-2ed518eafbd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv\" (UID: \"220fcd47-c1db-4446-b47e-2ed518eafbd6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:18.719621 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.719474    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgmcx\" (UniqueName: \"kubernetes.io/projected/220fcd47-c1db-4446-b47e-2ed518eafbd6-kube-api-access-fgmcx\") pod \"payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv\" (UID: \"220fcd47-c1db-4446-b47e-2ed518eafbd6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:18.719621 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.719588    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s98jd\" (UniqueName: \"kubernetes.io/projected/55bd9dc1-e687-4801-b27f-a211a4861592-kube-api-access-s98jd\") pod \"payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j\" (UID: \"55bd9dc1-e687-4801-b27f-a211a4861592\") " pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:18.719621 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.719618    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55bd9dc1-e687-4801-b27f-a211a4861592-plugins-config-volume\") pod \"payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j\" (UID: \"55bd9dc1-e687-4801-b27f-a211a4861592\") " pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:18.820629 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.820594    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s98jd\" (UniqueName: \"kubernetes.io/projected/55bd9dc1-e687-4801-b27f-a211a4861592-kube-api-access-s98jd\") pod \"payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j\" (UID: \"55bd9dc1-e687-4801-b27f-a211a4861592\") " pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:18.820629 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.820634    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55bd9dc1-e687-4801-b27f-a211a4861592-plugins-config-volume\") pod \"payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j\" (UID: \"55bd9dc1-e687-4801-b27f-a211a4861592\") " pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:18.820820 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.820662    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/220fcd47-c1db-4446-b47e-2ed518eafbd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv\" (UID: \"220fcd47-c1db-4446-b47e-2ed518eafbd6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:18.820820 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.820704    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgmcx\" (UniqueName: \"kubernetes.io/projected/220fcd47-c1db-4446-b47e-2ed518eafbd6-kube-api-access-fgmcx\") pod \"payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv\" (UID: \"220fcd47-c1db-4446-b47e-2ed518eafbd6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:18.821408 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.821389    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/220fcd47-c1db-4446-b47e-2ed518eafbd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv\" (UID: \"220fcd47-c1db-4446-b47e-2ed518eafbd6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:18.821459 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.821398    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55bd9dc1-e687-4801-b27f-a211a4861592-plugins-config-volume\") pod \"payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j\" (UID: \"55bd9dc1-e687-4801-b27f-a211a4861592\") " pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:18.829942 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.829914    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgmcx\" (UniqueName: \"kubernetes.io/projected/220fcd47-c1db-4446-b47e-2ed518eafbd6-kube-api-access-fgmcx\") pod \"payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv\" (UID: \"220fcd47-c1db-4446-b47e-2ed518eafbd6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:18.830187 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.830169    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s98jd\" (UniqueName: \"kubernetes.io/projected/55bd9dc1-e687-4801-b27f-a211a4861592-kube-api-access-s98jd\") pod \"payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j\" (UID: \"55bd9dc1-e687-4801-b27f-a211a4861592\") " pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:18.904794 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.904719    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:18.948302 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:18.948267    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:19.058699 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.058629    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"]
Jul 24 23:21:19.061009 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:21:19.060979    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220fcd47_c1db_4446_b47e_2ed518eafbd6.slice/crio-416a31f7677beeff4498ca100d7ac23d12505b7490c8ba99f4ee93ecce14651a WatchSource:0}: Error finding container 416a31f7677beeff4498ca100d7ac23d12505b7490c8ba99f4ee93ecce14651a: Status 404 returned error can't find the container with id 416a31f7677beeff4498ca100d7ac23d12505b7490c8ba99f4ee93ecce14651a
Jul 24 23:21:19.142149 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.142125    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"]
Jul 24 23:21:19.143939 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:21:19.143914    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55bd9dc1_e687_4801_b27f_a211a4861592.slice/crio-4574e6c90fcee5545e5e29d5270fab3ed0f53d76616316935494970e07396d26 WatchSource:0}: Error finding container 4574e6c90fcee5545e5e29d5270fab3ed0f53d76616316935494970e07396d26: Status 404 returned error can't find the container with id 4574e6c90fcee5545e5e29d5270fab3ed0f53d76616316935494970e07396d26
Jul 24 23:21:19.321945 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.321905    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw" event={"ID":"b0183855-e59a-415f-92a7-31ba78086aa1","Type":"ContainerStarted","Data":"ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7"}
Jul 24 23:21:19.322132 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.321968    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw" event={"ID":"b0183855-e59a-415f-92a7-31ba78086aa1","Type":"ContainerStarted","Data":"f812fe191a8f515fdc772b31fc8451c3b9a99db2fa4a77b4ff08af2d3ccdf2a5"}
Jul 24 23:21:19.322132 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.322019    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:19.323450 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.323429    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j" event={"ID":"55bd9dc1-e687-4801-b27f-a211a4861592","Type":"ContainerStarted","Data":"06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04"}
Jul 24 23:21:19.323562 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.323465    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j" event={"ID":"55bd9dc1-e687-4801-b27f-a211a4861592","Type":"ContainerStarted","Data":"4574e6c90fcee5545e5e29d5270fab3ed0f53d76616316935494970e07396d26"}
Jul 24 23:21:19.323562 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.323493    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:19.324955 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.324939    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv" event={"ID":"220fcd47-c1db-4446-b47e-2ed518eafbd6","Type":"ContainerStarted","Data":"02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875"}
Jul 24 23:21:19.325019 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.324970    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv" event={"ID":"220fcd47-c1db-4446-b47e-2ed518eafbd6","Type":"ContainerStarted","Data":"416a31f7677beeff4498ca100d7ac23d12505b7490c8ba99f4ee93ecce14651a"}
Jul 24 23:21:19.325019 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.325004    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:19.340320 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.340278    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw" podStartSLOduration=1.590228138 podStartE2EDuration="2.340264698s" totalImagesPullingTime="750.03656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:21:19.339111316 +0000 UTC m=+1736.616144907" watchObservedRunningTime="2026-07-24 23:21:19.340264698 +0000 UTC m=+1736.617298602"
Jul 24 23:21:19.358074 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.358025    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv" podStartSLOduration=1.358009434 podStartE2EDuration="1.358009434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:21:19.355521526 +0000 UTC m=+1736.632555093" watchObservedRunningTime="2026-07-24 23:21:19.358009434 +0000 UTC m=+1736.635042978"
Jul 24 23:21:19.376273 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:19.376197    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j" podStartSLOduration=1.376170514 podStartE2EDuration="1.376170514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:21:19.37134963 +0000 UTC m=+1736.648383192" watchObservedRunningTime="2026-07-24 23:21:19.376170514 +0000 UTC m=+1736.653204059"
Jul 24 23:21:25.335774 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:25.335740    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:30.331865 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:30.331826    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:30.332360 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:30.332042    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:53.113184 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.113152    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"]
Jul 24 23:21:53.113815 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.113397    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw" podUID="b0183855-e59a-415f-92a7-31ba78086aa1" containerName="maas-api" containerID="cri-o://ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7" gracePeriod=30
Jul 24 23:21:53.169436 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.169409    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"]
Jul 24 23:21:53.169681 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.169641    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j" podUID="55bd9dc1-e687-4801-b27f-a211a4861592" containerName="payload-processing" containerID="cri-o://06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04" gracePeriod=30
Jul 24 23:21:53.176772 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.176732    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"]
Jul 24 23:21:53.177001 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.176957    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv" podUID="220fcd47-c1db-4446-b47e-2ed518eafbd6" containerName="payload-pre-processing" containerID="cri-o://02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875" gracePeriod=30
Jul 24 23:21:53.416309 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.416237    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:53.435570 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.435537    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s49vv\" (UniqueName: \"kubernetes.io/projected/b0183855-e59a-415f-92a7-31ba78086aa1-kube-api-access-s49vv\") pod \"b0183855-e59a-415f-92a7-31ba78086aa1\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") "
Jul 24 23:21:53.435702 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.435624    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0183855-e59a-415f-92a7-31ba78086aa1-maas-api-tls\") pod \"b0183855-e59a-415f-92a7-31ba78086aa1\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") "
Jul 24 23:21:53.435775 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.435729    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0183855-e59a-415f-92a7-31ba78086aa1-openshift-service-ca\") pod \"b0183855-e59a-415f-92a7-31ba78086aa1\" (UID: \"b0183855-e59a-415f-92a7-31ba78086aa1\") "
Jul 24 23:21:53.436823 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.436792    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0183855-e59a-415f-92a7-31ba78086aa1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0183855-e59a-415f-92a7-31ba78086aa1" (UID: "b0183855-e59a-415f-92a7-31ba78086aa1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:21:53.441668 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.441637    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0183855-e59a-415f-92a7-31ba78086aa1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0183855-e59a-415f-92a7-31ba78086aa1" (UID: "b0183855-e59a-415f-92a7-31ba78086aa1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:21:53.441774 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.441718    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0183855-e59a-415f-92a7-31ba78086aa1-kube-api-access-s49vv" (OuterVolumeSpecName: "kube-api-access-s49vv") pod "b0183855-e59a-415f-92a7-31ba78086aa1" (UID: "b0183855-e59a-415f-92a7-31ba78086aa1"). InnerVolumeSpecName "kube-api-access-s49vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:21:53.443434 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.443415    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:53.465957 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.465936    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:53.485224 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.485104    2568 generic.go:356] "Generic (PLEG): container finished" podID="55bd9dc1-e687-4801-b27f-a211a4861592" containerID="06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04" exitCode=0
Jul 24 23:21:53.485224 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.485141    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j" event={"ID":"55bd9dc1-e687-4801-b27f-a211a4861592","Type":"ContainerDied","Data":"06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04"}
Jul 24 23:21:53.485224 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.485154    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"
Jul 24 23:21:53.485224 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.485193    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j" event={"ID":"55bd9dc1-e687-4801-b27f-a211a4861592","Type":"ContainerDied","Data":"4574e6c90fcee5545e5e29d5270fab3ed0f53d76616316935494970e07396d26"}
Jul 24 23:21:53.485224 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.485226    2568 scope.go:117] "RemoveContainer" containerID="06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04"
Jul 24 23:21:53.487401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.487364    2568 generic.go:356] "Generic (PLEG): container finished" podID="220fcd47-c1db-4446-b47e-2ed518eafbd6" containerID="02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875" exitCode=0
Jul 24 23:21:53.487401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.487398    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"
Jul 24 23:21:53.487696 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.487537    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv" event={"ID":"220fcd47-c1db-4446-b47e-2ed518eafbd6","Type":"ContainerDied","Data":"02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875"}
Jul 24 23:21:53.487696 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.487576    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv" event={"ID":"220fcd47-c1db-4446-b47e-2ed518eafbd6","Type":"ContainerDied","Data":"416a31f7677beeff4498ca100d7ac23d12505b7490c8ba99f4ee93ecce14651a"}
Jul 24 23:21:53.488860 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.488840    2568 generic.go:356] "Generic (PLEG): container finished" podID="b0183855-e59a-415f-92a7-31ba78086aa1" containerID="ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7" exitCode=0
Jul 24 23:21:53.488934 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.488880    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw" event={"ID":"b0183855-e59a-415f-92a7-31ba78086aa1","Type":"ContainerDied","Data":"ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7"}
Jul 24 23:21:53.488934 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.488902    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"
Jul 24 23:21:53.488934 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.488917    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw" event={"ID":"b0183855-e59a-415f-92a7-31ba78086aa1","Type":"ContainerDied","Data":"f812fe191a8f515fdc772b31fc8451c3b9a99db2fa4a77b4ff08af2d3ccdf2a5"}
Jul 24 23:21:53.498170 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.498151    2568 scope.go:117] "RemoveContainer" containerID="06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04"
Jul 24 23:21:53.498519 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:21:53.498498    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04\": container with ID starting with 06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04 not found: ID does not exist" containerID="06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04"
Jul 24 23:21:53.498615 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.498527    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04"} err="failed to get container status \"06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04\": rpc error: code = NotFound desc = could not find container \"06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04\": container with ID starting with 06187432bd1d8ed236455fd23f1b91876a480377ef2c19ce4bb8263344d42c04 not found: ID does not exist"
Jul 24 23:21:53.498615 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.498546    2568 scope.go:117] "RemoveContainer" containerID="02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875"
Jul 24 23:21:53.507168 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.507151    2568 scope.go:117] "RemoveContainer" containerID="02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875"
Jul 24 23:21:53.507483 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:21:53.507461    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875\": container with ID starting with 02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875 not found: ID does not exist" containerID="02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875"
Jul 24 23:21:53.507577 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.507490    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875"} err="failed to get container status \"02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875\": rpc error: code = NotFound desc = could not find container \"02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875\": container with ID starting with 02a3802ae1bada5ebc512ececc19ba42730f6f4d5f7e128334506915c3148875 not found: ID does not exist"
Jul 24 23:21:53.507577 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.507508    2568 scope.go:117] "RemoveContainer" containerID="ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7"
Jul 24 23:21:53.513451 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.513427    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"]
Jul 24 23:21:53.516546 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.516529    2568 scope.go:117] "RemoveContainer" containerID="ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7"
Jul 24 23:21:53.516827 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:21:53.516806    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7\": container with ID starting with ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7 not found: ID does not exist" containerID="ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7"
Jul 24 23:21:53.516904 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.516837    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7"} err="failed to get container status \"ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7\": rpc error: code = NotFound desc = could not find container \"ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7\": container with ID starting with ec32c60725233c44af4754d32ecd0de25aabdbb61565bb602a7d0399cb7d41d7 not found: ID does not exist"
Jul 24 23:21:53.517231 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.517214    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4bebff3-79b9cf5d44-jh7kw"]
Jul 24 23:21:53.537241 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.537221    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgmcx\" (UniqueName: \"kubernetes.io/projected/220fcd47-c1db-4446-b47e-2ed518eafbd6-kube-api-access-fgmcx\") pod \"220fcd47-c1db-4446-b47e-2ed518eafbd6\" (UID: \"220fcd47-c1db-4446-b47e-2ed518eafbd6\") "
Jul 24 23:21:53.537401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.537267    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/220fcd47-c1db-4446-b47e-2ed518eafbd6-plugins-config-volume\") pod \"220fcd47-c1db-4446-b47e-2ed518eafbd6\" (UID: \"220fcd47-c1db-4446-b47e-2ed518eafbd6\") "
Jul 24 23:21:53.537401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.537323    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s98jd\" (UniqueName: \"kubernetes.io/projected/55bd9dc1-e687-4801-b27f-a211a4861592-kube-api-access-s98jd\") pod \"55bd9dc1-e687-4801-b27f-a211a4861592\" (UID: \"55bd9dc1-e687-4801-b27f-a211a4861592\") "
Jul 24 23:21:53.537401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.537381    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55bd9dc1-e687-4801-b27f-a211a4861592-plugins-config-volume\") pod \"55bd9dc1-e687-4801-b27f-a211a4861592\" (UID: \"55bd9dc1-e687-4801-b27f-a211a4861592\") "
Jul 24 23:21:53.537617 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.537592    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/220fcd47-c1db-4446-b47e-2ed518eafbd6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "220fcd47-c1db-4446-b47e-2ed518eafbd6" (UID: "220fcd47-c1db-4446-b47e-2ed518eafbd6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:21:53.537727 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.537617    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0183855-e59a-415f-92a7-31ba78086aa1-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:53.537727 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.537638    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s49vv\" (UniqueName: \"kubernetes.io/projected/b0183855-e59a-415f-92a7-31ba78086aa1-kube-api-access-s49vv\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:53.537727 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.537653    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0183855-e59a-415f-92a7-31ba78086aa1-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:53.537849 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.537736    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55bd9dc1-e687-4801-b27f-a211a4861592-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55bd9dc1-e687-4801-b27f-a211a4861592" (UID: "55bd9dc1-e687-4801-b27f-a211a4861592"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:21:53.539235 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.539215    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220fcd47-c1db-4446-b47e-2ed518eafbd6-kube-api-access-fgmcx" (OuterVolumeSpecName: "kube-api-access-fgmcx") pod "220fcd47-c1db-4446-b47e-2ed518eafbd6" (UID: "220fcd47-c1db-4446-b47e-2ed518eafbd6"). InnerVolumeSpecName "kube-api-access-fgmcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:21:53.539397 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.539379    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55bd9dc1-e687-4801-b27f-a211a4861592-kube-api-access-s98jd" (OuterVolumeSpecName: "kube-api-access-s98jd") pod "55bd9dc1-e687-4801-b27f-a211a4861592" (UID: "55bd9dc1-e687-4801-b27f-a211a4861592"). InnerVolumeSpecName "kube-api-access-s98jd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:21:53.638150 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.638049    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgmcx\" (UniqueName: \"kubernetes.io/projected/220fcd47-c1db-4446-b47e-2ed518eafbd6-kube-api-access-fgmcx\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:53.638150 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.638093    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/220fcd47-c1db-4446-b47e-2ed518eafbd6-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:53.638150 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.638104    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s98jd\" (UniqueName: \"kubernetes.io/projected/55bd9dc1-e687-4801-b27f-a211a4861592-kube-api-access-s98jd\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:53.638150 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.638113    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55bd9dc1-e687-4801-b27f-a211a4861592-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:21:53.809325 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.809290    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"]
Jul 24 23:21:53.815057 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.815034    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a4bebff3-594c78d6b6-w8f8j"]
Jul 24 23:21:53.825392 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.825365    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"]
Jul 24 23:21:53.829269 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:53.829220    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a4bebff3-7859bb76d-bfclv"]
Jul 24 23:21:55.258305 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:55.258243    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="220fcd47-c1db-4446-b47e-2ed518eafbd6" path="/var/lib/kubelet/pods/220fcd47-c1db-4446-b47e-2ed518eafbd6/volumes"
Jul 24 23:21:55.258676 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:55.258636    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55bd9dc1-e687-4801-b27f-a211a4861592" path="/var/lib/kubelet/pods/55bd9dc1-e687-4801-b27f-a211a4861592/volumes"
Jul 24 23:21:55.258978 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:21:55.258965    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0183855-e59a-415f-92a7-31ba78086aa1" path="/var/lib/kubelet/pods/b0183855-e59a-415f-92a7-31ba78086aa1/volumes"
Jul 24 23:23:39.132129 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.132092    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"]
Jul 24 23:23:39.132628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.132506    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220fcd47-c1db-4446-b47e-2ed518eafbd6" containerName="payload-pre-processing"
Jul 24 23:23:39.132628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.132519    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="220fcd47-c1db-4446-b47e-2ed518eafbd6" containerName="payload-pre-processing"
Jul 24 23:23:39.132628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.132528    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55bd9dc1-e687-4801-b27f-a211a4861592" containerName="payload-processing"
Jul 24 23:23:39.132628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.132533    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="55bd9dc1-e687-4801-b27f-a211a4861592" containerName="payload-processing"
Jul 24 23:23:39.132628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.132551    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0183855-e59a-415f-92a7-31ba78086aa1" containerName="maas-api"
Jul 24 23:23:39.132628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.132557    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0183855-e59a-415f-92a7-31ba78086aa1" containerName="maas-api"
Jul 24 23:23:39.132628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.132609    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="55bd9dc1-e687-4801-b27f-a211a4861592" containerName="payload-processing"
Jul 24 23:23:39.132628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.132619    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="220fcd47-c1db-4446-b47e-2ed518eafbd6" containerName="payload-pre-processing"
Jul 24 23:23:39.132628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.132626    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0183855-e59a-415f-92a7-31ba78086aa1" containerName="maas-api"
Jul 24 23:23:39.135856 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.135840    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.138628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.138608    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9b1bceef-serving-cert\""
Jul 24 23:23:39.151264 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.151227    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"]
Jul 24 23:23:39.225929 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.225892    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/805da87e-8ade-4703-a12c-9ef37af94f76-openshift-service-ca\") pod \"maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.225929 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.225935    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9smqz\" (UniqueName: \"kubernetes.io/projected/805da87e-8ade-4703-a12c-9ef37af94f76-kube-api-access-9smqz\") pod \"maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.226163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.225977    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805da87e-8ade-4703-a12c-9ef37af94f76-maas-api-tls\") pod \"maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.327128 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.327085    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/805da87e-8ade-4703-a12c-9ef37af94f76-openshift-service-ca\") pod \"maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.327369 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.327145    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9smqz\" (UniqueName: \"kubernetes.io/projected/805da87e-8ade-4703-a12c-9ef37af94f76-kube-api-access-9smqz\") pod \"maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.327369 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.327216    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805da87e-8ade-4703-a12c-9ef37af94f76-maas-api-tls\") pod \"maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.327494 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:23:39.327381    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-serving-cert: secret "maas-api-e2e-mt-9b1bceef-serving-cert" not found
Jul 24 23:23:39.327494 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:23:39.327488    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/805da87e-8ade-4703-a12c-9ef37af94f76-maas-api-tls podName:805da87e-8ade-4703-a12c-9ef37af94f76 nodeName:}" failed. No retries permitted until 2026-07-24 23:23:39.827464114 +0000 UTC m=+1877.104497640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/805da87e-8ade-4703-a12c-9ef37af94f76-maas-api-tls") pod "maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm" (UID: "805da87e-8ade-4703-a12c-9ef37af94f76") : secret "maas-api-e2e-mt-9b1bceef-serving-cert" not found
Jul 24 23:23:39.327884 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.327867    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/805da87e-8ade-4703-a12c-9ef37af94f76-openshift-service-ca\") pod \"maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.337008 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.336981    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9smqz\" (UniqueName: \"kubernetes.io/projected/805da87e-8ade-4703-a12c-9ef37af94f76-kube-api-access-9smqz\") pod \"maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.831947 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.831916    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805da87e-8ade-4703-a12c-9ef37af94f76-maas-api-tls\") pod \"maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.834453 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.834428    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805da87e-8ade-4703-a12c-9ef37af94f76-maas-api-tls\") pod \"maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:39.980971 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.980938    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"]
Jul 24 23:23:39.984707 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.984685    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:23:39.987266 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.987224    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9b1bceef\""
Jul 24 23:23:39.987392 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.987302    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9b1bceef-dockercfg-smgs7\""
Jul 24 23:23:39.995399 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:39.995376    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"]
Jul 24 23:23:40.046036 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.046003    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:40.134915 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.134885    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2n9f\" (UniqueName: \"kubernetes.io/projected/91f50881-51f5-4d81-9714-c38da28706b9-kube-api-access-h2n9f\") pod \"payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h\" (UID: \"91f50881-51f5-4d81-9714-c38da28706b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:23:40.135346 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.134993    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91f50881-51f5-4d81-9714-c38da28706b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h\" (UID: \"91f50881-51f5-4d81-9714-c38da28706b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:23:40.235751 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.235701    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2n9f\" (UniqueName: \"kubernetes.io/projected/91f50881-51f5-4d81-9714-c38da28706b9-kube-api-access-h2n9f\") pod \"payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h\" (UID: \"91f50881-51f5-4d81-9714-c38da28706b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:23:40.235958 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.235799    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91f50881-51f5-4d81-9714-c38da28706b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h\" (UID: \"91f50881-51f5-4d81-9714-c38da28706b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:23:40.236473 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.236453    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91f50881-51f5-4d81-9714-c38da28706b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h\" (UID: \"91f50881-51f5-4d81-9714-c38da28706b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:23:40.246463 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.246438    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2n9f\" (UniqueName: \"kubernetes.io/projected/91f50881-51f5-4d81-9714-c38da28706b9-kube-api-access-h2n9f\") pod \"payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h\" (UID: \"91f50881-51f5-4d81-9714-c38da28706b9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:23:40.295547 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.295510    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:23:40.391126 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.391092    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"]
Jul 24 23:23:40.393735 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:23:40.393703    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805da87e_8ade_4703_a12c_9ef37af94f76.slice/crio-56f7e3afabe1af64ea8e19dc42514200dd943b94630753c623c0b0b7f1232d1c WatchSource:0}: Error finding container 56f7e3afabe1af64ea8e19dc42514200dd943b94630753c623c0b0b7f1232d1c: Status 404 returned error can't find the container with id 56f7e3afabe1af64ea8e19dc42514200dd943b94630753c623c0b0b7f1232d1c
Jul 24 23:23:40.435435 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.434241    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"]
Jul 24 23:23:40.438056 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:23:40.438015    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f50881_51f5_4d81_9714_c38da28706b9.slice/crio-13870f5c55e50276bd377680cb98a1617edc29a6823d139f0a52be4662d04c27 WatchSource:0}: Error finding container 13870f5c55e50276bd377680cb98a1617edc29a6823d139f0a52be4662d04c27: Status 404 returned error can't find the container with id 13870f5c55e50276bd377680cb98a1617edc29a6823d139f0a52be4662d04c27
Jul 24 23:23:40.988881 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.988847    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm" event={"ID":"805da87e-8ade-4703-a12c-9ef37af94f76","Type":"ContainerStarted","Data":"56f7e3afabe1af64ea8e19dc42514200dd943b94630753c623c0b0b7f1232d1c"}
Jul 24 23:23:40.990395 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.990370    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h" event={"ID":"91f50881-51f5-4d81-9714-c38da28706b9","Type":"ContainerStarted","Data":"aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e"}
Jul 24 23:23:40.990521 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.990412    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h" event={"ID":"91f50881-51f5-4d81-9714-c38da28706b9","Type":"ContainerStarted","Data":"13870f5c55e50276bd377680cb98a1617edc29a6823d139f0a52be4662d04c27"}
Jul 24 23:23:40.990521 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:40.990505    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:23:41.006057 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:41.006015    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h" podStartSLOduration=2.005996761 podStartE2EDuration="2.005996761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:23:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:23:41.004460224 +0000 UTC m=+1878.281493789" watchObservedRunningTime="2026-07-24 23:23:41.005996761 +0000 UTC m=+1878.283030304"
Jul 24 23:23:41.997215 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:41.997161    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm" event={"ID":"805da87e-8ade-4703-a12c-9ef37af94f76","Type":"ContainerStarted","Data":"68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75"}
Jul 24 23:23:41.997636 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:41.997285    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:42.015398 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:42.015350    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm" podStartSLOduration=2.460961153 podStartE2EDuration="3.015334311s" totalImagesPullingTime="554.373158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:23:42.013496004 +0000 UTC m=+1879.290529559" watchObservedRunningTime="2026-07-24 23:23:42.015334311 +0000 UTC m=+1879.292367855"
Jul 24 23:23:48.007849 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:48.007821    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:23:50.979508 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:50.979472    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"]
Jul 24 23:23:50.982976 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:50.982960    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"
Jul 24 23:23:50.985367 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:50.985345    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:23:50.991803 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:50.991777    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"]
Jul 24 23:23:51.143360 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:51.143318    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd0c33c-4970-4648-ae0b-f4378599acd2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm\" (UID: \"5bd0c33c-4970-4648-ae0b-f4378599acd2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"
Jul 24 23:23:51.244033 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:51.243951    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd0c33c-4970-4648-ae0b-f4378599acd2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm\" (UID: \"5bd0c33c-4970-4648-ae0b-f4378599acd2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"
Jul 24 23:23:51.244590 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:51.244565    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd0c33c-4970-4648-ae0b-f4378599acd2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm\" (UID: \"5bd0c33c-4970-4648-ae0b-f4378599acd2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"
Jul 24 23:23:51.294124 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:51.294094    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"
Jul 24 23:23:51.418834 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:51.418810    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"]
Jul 24 23:23:51.420971 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:23:51.420942    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd0c33c_4970_4648_ae0b_f4378599acd2.slice/crio-dca44297f778cb0a4b538d21d0e8f4f533184e70dfee52d3fb06325ad341e06a WatchSource:0}: Error finding container dca44297f778cb0a4b538d21d0e8f4f533184e70dfee52d3fb06325ad341e06a: Status 404 returned error can't find the container with id dca44297f778cb0a4b538d21d0e8f4f533184e70dfee52d3fb06325ad341e06a
Jul 24 23:23:51.999677 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:51.999652    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:23:52.054369 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:52.054332    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm" event={"ID":"5bd0c33c-4970-4648-ae0b-f4378599acd2","Type":"ContainerStarted","Data":"c0f636c45ecb98f80ac79e22ed9b7655d2bf3a4052ef5238c28662d5a78bd9cb"}
Jul 24 23:23:52.054560 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:52.054416    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm" event={"ID":"5bd0c33c-4970-4648-ae0b-f4378599acd2","Type":"ContainerStarted","Data":"dca44297f778cb0a4b538d21d0e8f4f533184e70dfee52d3fb06325ad341e06a"}
Jul 24 23:23:52.072546 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:52.072500    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm" podStartSLOduration=2.072486555 podStartE2EDuration="2.072486555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:23:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:23:52.070852424 +0000 UTC m=+1889.347885968" watchObservedRunningTime="2026-07-24 23:23:52.072486555 +0000 UTC m=+1889.349520099"
Jul 24 23:23:53.060554 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:53.060521    2568 generic.go:356] "Generic (PLEG): container finished" podID="5bd0c33c-4970-4648-ae0b-f4378599acd2" containerID="c0f636c45ecb98f80ac79e22ed9b7655d2bf3a4052ef5238c28662d5a78bd9cb" exitCode=0
Jul 24 23:23:53.060958 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:53.060606    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm" event={"ID":"5bd0c33c-4970-4648-ae0b-f4378599acd2","Type":"ContainerDied","Data":"c0f636c45ecb98f80ac79e22ed9b7655d2bf3a4052ef5238c28662d5a78bd9cb"}
Jul 24 23:23:54.193398 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:54.193377    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"
Jul 24 23:23:54.374629 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:54.374540    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd0c33c-4970-4648-ae0b-f4378599acd2-maas-api-service-ca\") pod \"5bd0c33c-4970-4648-ae0b-f4378599acd2\" (UID: \"5bd0c33c-4970-4648-ae0b-f4378599acd2\") "
Jul 24 23:23:54.374926 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:54.374890    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bd0c33c-4970-4648-ae0b-f4378599acd2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5bd0c33c-4970-4648-ae0b-f4378599acd2" (UID: "5bd0c33c-4970-4648-ae0b-f4378599acd2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:23:54.475966 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:54.475935    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd0c33c-4970-4648-ae0b-f4378599acd2-maas-api-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:23:55.071572 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:55.071542    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"
Jul 24 23:23:55.071761 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:55.071579    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm" event={"ID":"5bd0c33c-4970-4648-ae0b-f4378599acd2","Type":"ContainerDied","Data":"dca44297f778cb0a4b538d21d0e8f4f533184e70dfee52d3fb06325ad341e06a"}
Jul 24 23:23:55.071761 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:23:55.071633    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dca44297f778cb0a4b538d21d0e8f4f533184e70dfee52d3fb06325ad341e06a"
Jul 24 23:24:01.101673 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.101642    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"]
Jul 24 23:24:01.102164 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.101833    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm" podUID="805da87e-8ade-4703-a12c-9ef37af94f76" containerName="maas-api" containerID="cri-o://68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75" gracePeriod=30
Jul 24 23:24:01.162112 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.162080    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"]
Jul 24 23:24:01.162366 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.162316    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h" podUID="91f50881-51f5-4d81-9714-c38da28706b9" containerName="payload-pre-processing" containerID="cri-o://aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e" gracePeriod=30
Jul 24 23:24:01.409836 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.409795    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:24:01.433656 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.433629    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805da87e-8ade-4703-a12c-9ef37af94f76-maas-api-tls\") pod \"805da87e-8ade-4703-a12c-9ef37af94f76\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") "
Jul 24 23:24:01.433806 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.433689    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/805da87e-8ade-4703-a12c-9ef37af94f76-openshift-service-ca\") pod \"805da87e-8ade-4703-a12c-9ef37af94f76\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") "
Jul 24 23:24:01.433806 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.433728    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9smqz\" (UniqueName: \"kubernetes.io/projected/805da87e-8ade-4703-a12c-9ef37af94f76-kube-api-access-9smqz\") pod \"805da87e-8ade-4703-a12c-9ef37af94f76\" (UID: \"805da87e-8ade-4703-a12c-9ef37af94f76\") "
Jul 24 23:24:01.434219 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.434171    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/805da87e-8ade-4703-a12c-9ef37af94f76-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "805da87e-8ade-4703-a12c-9ef37af94f76" (UID: "805da87e-8ade-4703-a12c-9ef37af94f76"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:24:01.434854 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.434813    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:24:01.436447 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.436413    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/805da87e-8ade-4703-a12c-9ef37af94f76-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "805da87e-8ade-4703-a12c-9ef37af94f76" (UID: "805da87e-8ade-4703-a12c-9ef37af94f76"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:24:01.436588 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.436569    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805da87e-8ade-4703-a12c-9ef37af94f76-kube-api-access-9smqz" (OuterVolumeSpecName: "kube-api-access-9smqz") pod "805da87e-8ade-4703-a12c-9ef37af94f76" (UID: "805da87e-8ade-4703-a12c-9ef37af94f76"). InnerVolumeSpecName "kube-api-access-9smqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:24:01.534729 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.534701    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2n9f\" (UniqueName: \"kubernetes.io/projected/91f50881-51f5-4d81-9714-c38da28706b9-kube-api-access-h2n9f\") pod \"91f50881-51f5-4d81-9714-c38da28706b9\" (UID: \"91f50881-51f5-4d81-9714-c38da28706b9\") "
Jul 24 23:24:01.534893 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.534751    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91f50881-51f5-4d81-9714-c38da28706b9-plugins-config-volume\") pod \"91f50881-51f5-4d81-9714-c38da28706b9\" (UID: \"91f50881-51f5-4d81-9714-c38da28706b9\") "
Jul 24 23:24:01.534945 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.534930    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/805da87e-8ade-4703-a12c-9ef37af94f76-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:24:01.534982 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.534951    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/805da87e-8ade-4703-a12c-9ef37af94f76-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:24:01.534982 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.534961    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9smqz\" (UniqueName: \"kubernetes.io/projected/805da87e-8ade-4703-a12c-9ef37af94f76-kube-api-access-9smqz\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:24:01.535173 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.535152    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91f50881-51f5-4d81-9714-c38da28706b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "91f50881-51f5-4d81-9714-c38da28706b9" (UID: "91f50881-51f5-4d81-9714-c38da28706b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:24:01.536810 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.536787    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f50881-51f5-4d81-9714-c38da28706b9-kube-api-access-h2n9f" (OuterVolumeSpecName: "kube-api-access-h2n9f") pod "91f50881-51f5-4d81-9714-c38da28706b9" (UID: "91f50881-51f5-4d81-9714-c38da28706b9"). InnerVolumeSpecName "kube-api-access-h2n9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:24:01.635564 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.635488    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2n9f\" (UniqueName: \"kubernetes.io/projected/91f50881-51f5-4d81-9714-c38da28706b9-kube-api-access-h2n9f\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:24:01.635564 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:01.635514    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91f50881-51f5-4d81-9714-c38da28706b9-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:24:02.103948 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.103910    2568 generic.go:356] "Generic (PLEG): container finished" podID="91f50881-51f5-4d81-9714-c38da28706b9" containerID="aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e" exitCode=0
Jul 24 23:24:02.104391 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.103957    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"
Jul 24 23:24:02.104391 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.103996    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h" event={"ID":"91f50881-51f5-4d81-9714-c38da28706b9","Type":"ContainerDied","Data":"aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e"}
Jul 24 23:24:02.104391 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.104056    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h" event={"ID":"91f50881-51f5-4d81-9714-c38da28706b9","Type":"ContainerDied","Data":"13870f5c55e50276bd377680cb98a1617edc29a6823d139f0a52be4662d04c27"}
Jul 24 23:24:02.104391 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.104085    2568 scope.go:117] "RemoveContainer" containerID="aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e"
Jul 24 23:24:02.105505 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.105421    2568 generic.go:356] "Generic (PLEG): container finished" podID="805da87e-8ade-4703-a12c-9ef37af94f76" containerID="68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75" exitCode=0
Jul 24 23:24:02.105505 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.105458    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm" event={"ID":"805da87e-8ade-4703-a12c-9ef37af94f76","Type":"ContainerDied","Data":"68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75"}
Jul 24 23:24:02.105505 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.105474    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"
Jul 24 23:24:02.105505 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.105500    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm" event={"ID":"805da87e-8ade-4703-a12c-9ef37af94f76","Type":"ContainerDied","Data":"56f7e3afabe1af64ea8e19dc42514200dd943b94630753c623c0b0b7f1232d1c"}
Jul 24 23:24:02.121603 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.121588    2568 scope.go:117] "RemoveContainer" containerID="aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e"
Jul 24 23:24:02.121869 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:24:02.121851    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e\": container with ID starting with aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e not found: ID does not exist" containerID="aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e"
Jul 24 23:24:02.121944 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.121879    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e"} err="failed to get container status \"aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e\": rpc error: code = NotFound desc = could not find container \"aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e\": container with ID starting with aaab0f8a20d8f08551ee8a36e712d932b04736199cf1d1706d6d4faee54ec17e not found: ID does not exist"
Jul 24 23:24:02.121944 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.121900    2568 scope.go:117] "RemoveContainer" containerID="68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75"
Jul 24 23:24:02.129234 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.129218    2568 scope.go:117] "RemoveContainer" containerID="68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75"
Jul 24 23:24:02.129526 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:24:02.129501    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75\": container with ID starting with 68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75 not found: ID does not exist" containerID="68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75"
Jul 24 23:24:02.129581 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.129524    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75"} err="failed to get container status \"68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75\": rpc error: code = NotFound desc = could not find container \"68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75\": container with ID starting with 68ad1bf38f2b014d0fd76ae336d7b2aef7b91351751dd82eae3df2e7a9074a75 not found: ID does not exist"
Jul 24 23:24:02.139509 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.139489    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"]
Jul 24 23:24:02.146423 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.146402    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9b1bceef-8567f887dd-6gq6h"]
Jul 24 23:24:02.158295 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.158275    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"]
Jul 24 23:24:02.163445 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:02.163426    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b1bceef-5cd699c686-g6xcm"]
Jul 24 23:24:03.257944 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:03.257904    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="805da87e-8ade-4703-a12c-9ef37af94f76" path="/var/lib/kubelet/pods/805da87e-8ade-4703-a12c-9ef37af94f76/volumes"
Jul 24 23:24:03.258487 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:03.258467    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91f50881-51f5-4d81-9714-c38da28706b9" path="/var/lib/kubelet/pods/91f50881-51f5-4d81-9714-c38da28706b9/volumes"
Jul 24 23:24:06.068810 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:06.068775    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"]
Jul 24 23:24:06.073448 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:06.073417    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b1bceef-m2mvm"]
Jul 24 23:24:07.257411 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:07.257380    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bd0c33c-4970-4648-ae0b-f4378599acd2" path="/var/lib/kubelet/pods/5bd0c33c-4970-4648-ae0b-f4378599acd2/volumes"
Jul 24 23:24:35.582869 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.582833    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"]
Jul 24 23:24:35.583344 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.583189    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805da87e-8ade-4703-a12c-9ef37af94f76" containerName="maas-api"
Jul 24 23:24:35.583344 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.583200    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="805da87e-8ade-4703-a12c-9ef37af94f76" containerName="maas-api"
Jul 24 23:24:35.583344 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.583216    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd0c33c-4970-4648-ae0b-f4378599acd2" containerName="revoke-keys"
Jul 24 23:24:35.583344 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.583221    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd0c33c-4970-4648-ae0b-f4378599acd2" containerName="revoke-keys"
Jul 24 23:24:35.583344 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.583229    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91f50881-51f5-4d81-9714-c38da28706b9" containerName="payload-pre-processing"
Jul 24 23:24:35.583344 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.583235    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="91f50881-51f5-4d81-9714-c38da28706b9" containerName="payload-pre-processing"
Jul 24 23:24:35.583344 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.583307    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="805da87e-8ade-4703-a12c-9ef37af94f76" containerName="maas-api"
Jul 24 23:24:35.583344 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.583316    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="91f50881-51f5-4d81-9714-c38da28706b9" containerName="payload-pre-processing"
Jul 24 23:24:35.583344 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.583326    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bd0c33c-4970-4648-ae0b-f4378599acd2" containerName="revoke-keys"
Jul 24 23:24:35.586134 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.586115    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:35.588987 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.588958    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e9d44355-serving-cert\""
Jul 24 23:24:35.600070 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.600047    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"]
Jul 24 23:24:35.736324 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.736283    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8dcf65b-8d86-4217-b1af-92a423040668-maas-api-tls\") pod \"maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:35.736495 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.736349    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4vzh\" (UniqueName: \"kubernetes.io/projected/a8dcf65b-8d86-4217-b1af-92a423040668-kube-api-access-z4vzh\") pod \"maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:35.736495 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.736404    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8dcf65b-8d86-4217-b1af-92a423040668-openshift-service-ca\") pod \"maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:35.837502 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.837421    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4vzh\" (UniqueName: \"kubernetes.io/projected/a8dcf65b-8d86-4217-b1af-92a423040668-kube-api-access-z4vzh\") pod \"maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:35.837502 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.837464    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8dcf65b-8d86-4217-b1af-92a423040668-openshift-service-ca\") pod \"maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:35.837730 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.837519    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8dcf65b-8d86-4217-b1af-92a423040668-maas-api-tls\") pod \"maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:35.837730 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:24:35.837610    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-serving-cert: secret "maas-api-e2e-mt-e9d44355-serving-cert" not found
Jul 24 23:24:35.837730 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:24:35.837662    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8dcf65b-8d86-4217-b1af-92a423040668-maas-api-tls podName:a8dcf65b-8d86-4217-b1af-92a423040668 nodeName:}" failed. No retries permitted until 2026-07-24 23:24:36.337645603 +0000 UTC m=+1933.614679125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a8dcf65b-8d86-4217-b1af-92a423040668-maas-api-tls") pod "maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5" (UID: "a8dcf65b-8d86-4217-b1af-92a423040668") : secret "maas-api-e2e-mt-e9d44355-serving-cert" not found
Jul 24 23:24:35.838287 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.838236    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8dcf65b-8d86-4217-b1af-92a423040668-openshift-service-ca\") pod \"maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:35.846663 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:35.846641    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4vzh\" (UniqueName: \"kubernetes.io/projected/a8dcf65b-8d86-4217-b1af-92a423040668-kube-api-access-z4vzh\") pod \"maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:36.256519 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.256472    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"]
Jul 24 23:24:36.260100 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.260085    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:24:36.267203 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.267172    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e9d44355-dockercfg-4zx2h\""
Jul 24 23:24:36.267345 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.267326    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e9d44355\""
Jul 24 23:24:36.276858 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.276835    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"]
Jul 24 23:24:36.307534 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.307502    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"]
Jul 24 23:24:36.310936 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.310921    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:24:36.338155 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.338134    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"]
Jul 24 23:24:36.342720 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.342696    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef458705-b19a-47d0-a734-c989f57ce834-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4\" (UID: \"ef458705-b19a-47d0-a734-c989f57ce834\") " pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:24:36.342820 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.342729    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scjdx\" (UniqueName: \"kubernetes.io/projected/ef458705-b19a-47d0-a734-c989f57ce834-kube-api-access-scjdx\") pod \"payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4\" (UID: \"ef458705-b19a-47d0-a734-c989f57ce834\") " pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:24:36.342869 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.342843    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8dcf65b-8d86-4217-b1af-92a423040668-maas-api-tls\") pod \"maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:36.345046 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.345024    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8dcf65b-8d86-4217-b1af-92a423040668-maas-api-tls\") pod \"maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:36.443574 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.443541    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mldzc\" (UniqueName: \"kubernetes.io/projected/cf394dc2-adc9-4eb6-9105-5546a72724a4-kube-api-access-mldzc\") pod \"payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh\" (UID: \"cf394dc2-adc9-4eb6-9105-5546a72724a4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:24:36.443728 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.443595    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf394dc2-adc9-4eb6-9105-5546a72724a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh\" (UID: \"cf394dc2-adc9-4eb6-9105-5546a72724a4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:24:36.443728 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.443655    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef458705-b19a-47d0-a734-c989f57ce834-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4\" (UID: \"ef458705-b19a-47d0-a734-c989f57ce834\") " pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:24:36.443728 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.443691    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scjdx\" (UniqueName: \"kubernetes.io/projected/ef458705-b19a-47d0-a734-c989f57ce834-kube-api-access-scjdx\") pod \"payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4\" (UID: \"ef458705-b19a-47d0-a734-c989f57ce834\") " pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:24:36.444336 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.444319    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef458705-b19a-47d0-a734-c989f57ce834-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4\" (UID: \"ef458705-b19a-47d0-a734-c989f57ce834\") " pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:24:36.451885 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.451860    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scjdx\" (UniqueName: \"kubernetes.io/projected/ef458705-b19a-47d0-a734-c989f57ce834-kube-api-access-scjdx\") pod \"payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4\" (UID: \"ef458705-b19a-47d0-a734-c989f57ce834\") " pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:24:36.496864 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.496838    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:36.545265 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.545209    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mldzc\" (UniqueName: \"kubernetes.io/projected/cf394dc2-adc9-4eb6-9105-5546a72724a4-kube-api-access-mldzc\") pod \"payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh\" (UID: \"cf394dc2-adc9-4eb6-9105-5546a72724a4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:24:36.545445 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.545283    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf394dc2-adc9-4eb6-9105-5546a72724a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh\" (UID: \"cf394dc2-adc9-4eb6-9105-5546a72724a4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:24:36.546200 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.546141    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf394dc2-adc9-4eb6-9105-5546a72724a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh\" (UID: \"cf394dc2-adc9-4eb6-9105-5546a72724a4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:24:36.554055 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.554000    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mldzc\" (UniqueName: \"kubernetes.io/projected/cf394dc2-adc9-4eb6-9105-5546a72724a4-kube-api-access-mldzc\") pod \"payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh\" (UID: \"cf394dc2-adc9-4eb6-9105-5546a72724a4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:24:36.570178 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.570133    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:24:36.621204 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.621167    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:24:36.639651 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.639627    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"]
Jul 24 23:24:36.641664 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:24:36.641632    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8dcf65b_8d86_4217_b1af_92a423040668.slice/crio-db28faa6609ccbe1d81633940b134e36d46ee35f6d184c90fbe69c708720820f WatchSource:0}: Error finding container db28faa6609ccbe1d81633940b134e36d46ee35f6d184c90fbe69c708720820f: Status 404 returned error can't find the container with id db28faa6609ccbe1d81633940b134e36d46ee35f6d184c90fbe69c708720820f
Jul 24 23:24:36.644021 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.644002    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:24:36.720471 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:24:36.720445    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef458705_b19a_47d0_a734_c989f57ce834.slice/crio-8a40b1bbfe883f827286e890b8ef0b4a8b9ecfd1d2bdedfaee4300fb1391de03 WatchSource:0}: Error finding container 8a40b1bbfe883f827286e890b8ef0b4a8b9ecfd1d2bdedfaee4300fb1391de03: Status 404 returned error can't find the container with id 8a40b1bbfe883f827286e890b8ef0b4a8b9ecfd1d2bdedfaee4300fb1391de03
Jul 24 23:24:36.720769 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.720746    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"]
Jul 24 23:24:36.762647 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:36.762627    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"]
Jul 24 23:24:36.771238 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:24:36.771216    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf394dc2_adc9_4eb6_9105_5546a72724a4.slice/crio-1dce97aa0da5bf78e62cb9eca425763c9e1c137057dc9dc032596111730f343a WatchSource:0}: Error finding container 1dce97aa0da5bf78e62cb9eca425763c9e1c137057dc9dc032596111730f343a: Status 404 returned error can't find the container with id 1dce97aa0da5bf78e62cb9eca425763c9e1c137057dc9dc032596111730f343a
Jul 24 23:24:37.284818 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.284781    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4" event={"ID":"ef458705-b19a-47d0-a734-c989f57ce834","Type":"ContainerStarted","Data":"e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758"}
Jul 24 23:24:37.285010 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.284841    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4" event={"ID":"ef458705-b19a-47d0-a734-c989f57ce834","Type":"ContainerStarted","Data":"8a40b1bbfe883f827286e890b8ef0b4a8b9ecfd1d2bdedfaee4300fb1391de03"}
Jul 24 23:24:37.285010 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.284874    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:24:37.286357 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.286327    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5" event={"ID":"a8dcf65b-8d86-4217-b1af-92a423040668","Type":"ContainerStarted","Data":"d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc"}
Jul 24 23:24:37.286494 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.286374    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5" event={"ID":"a8dcf65b-8d86-4217-b1af-92a423040668","Type":"ContainerStarted","Data":"db28faa6609ccbe1d81633940b134e36d46ee35f6d184c90fbe69c708720820f"}
Jul 24 23:24:37.286494 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.286413    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:37.287889 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.287867    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh" event={"ID":"cf394dc2-adc9-4eb6-9105-5546a72724a4","Type":"ContainerStarted","Data":"f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161"}
Jul 24 23:24:37.287996 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.287909    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:24:37.287996 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.287926    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh" event={"ID":"cf394dc2-adc9-4eb6-9105-5546a72724a4","Type":"ContainerStarted","Data":"1dce97aa0da5bf78e62cb9eca425763c9e1c137057dc9dc032596111730f343a"}
Jul 24 23:24:37.326533 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.326487    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4" podStartSLOduration=1.326474099 podStartE2EDuration="1.326474099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:24:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:24:37.311337071 +0000 UTC m=+1934.588370635" watchObservedRunningTime="2026-07-24 23:24:37.326474099 +0000 UTC m=+1934.603507642"
Jul 24 23:24:37.340797 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.340757    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh" podStartSLOduration=1.340744199 podStartE2EDuration="1.340744199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:24:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:24:37.339665722 +0000 UTC m=+1934.616699278" watchObservedRunningTime="2026-07-24 23:24:37.340744199 +0000 UTC m=+1934.617777744"
Jul 24 23:24:37.368719 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:37.368676    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5" podStartSLOduration=2.087966692 podStartE2EDuration="2.368663465s" totalImagesPullingTime="280.696773ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:24:37.366286322 +0000 UTC m=+1934.643319860" watchObservedRunningTime="2026-07-24 23:24:37.368663465 +0000 UTC m=+1934.645697008"
Jul 24 23:24:43.298044 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:43.298013    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:24:48.294835 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:48.294801    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:24:48.295279 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:24:48.294957    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:25:35.010916 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.010887    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"]
Jul 24 23:25:35.011415 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.011138    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5" podUID="a8dcf65b-8d86-4217-b1af-92a423040668" containerName="maas-api" containerID="cri-o://d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc" gracePeriod=30
Jul 24 23:25:35.059719 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.059693    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"]
Jul 24 23:25:35.059909 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.059889    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4" podUID="ef458705-b19a-47d0-a734-c989f57ce834" containerName="payload-processing" containerID="cri-o://e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758" gracePeriod=30
Jul 24 23:25:35.065300 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.065272    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"]
Jul 24 23:25:35.065546 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.065498    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh" podUID="cf394dc2-adc9-4eb6-9105-5546a72724a4" containerName="payload-pre-processing" containerID="cri-o://f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161" gracePeriod=30
Jul 24 23:25:35.328024 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.328000    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:25:35.331467 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.331451    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:25:35.352688 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.352663    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:25:35.354704 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.354686    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8dcf65b-8d86-4217-b1af-92a423040668-maas-api-tls\") pod \"a8dcf65b-8d86-4217-b1af-92a423040668\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") "
Jul 24 23:25:35.354773 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.354717    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef458705-b19a-47d0-a734-c989f57ce834-plugins-config-volume\") pod \"ef458705-b19a-47d0-a734-c989f57ce834\" (UID: \"ef458705-b19a-47d0-a734-c989f57ce834\") "
Jul 24 23:25:35.354816 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.354784    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8dcf65b-8d86-4217-b1af-92a423040668-openshift-service-ca\") pod \"a8dcf65b-8d86-4217-b1af-92a423040668\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") "
Jul 24 23:25:35.354816 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.354803    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4vzh\" (UniqueName: \"kubernetes.io/projected/a8dcf65b-8d86-4217-b1af-92a423040668-kube-api-access-z4vzh\") pod \"a8dcf65b-8d86-4217-b1af-92a423040668\" (UID: \"a8dcf65b-8d86-4217-b1af-92a423040668\") "
Jul 24 23:25:35.354875 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.354862    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scjdx\" (UniqueName: \"kubernetes.io/projected/ef458705-b19a-47d0-a734-c989f57ce834-kube-api-access-scjdx\") pod \"ef458705-b19a-47d0-a734-c989f57ce834\" (UID: \"ef458705-b19a-47d0-a734-c989f57ce834\") "
Jul 24 23:25:35.355197 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.355163    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8dcf65b-8d86-4217-b1af-92a423040668-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8dcf65b-8d86-4217-b1af-92a423040668" (UID: "a8dcf65b-8d86-4217-b1af-92a423040668"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:25:35.355304 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.355204    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef458705-b19a-47d0-a734-c989f57ce834-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef458705-b19a-47d0-a734-c989f57ce834" (UID: "ef458705-b19a-47d0-a734-c989f57ce834"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:25:35.355376 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.355351    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef458705-b19a-47d0-a734-c989f57ce834-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:25:35.355376 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.355372    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8dcf65b-8d86-4217-b1af-92a423040668-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:25:35.357210 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.357192    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8dcf65b-8d86-4217-b1af-92a423040668-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8dcf65b-8d86-4217-b1af-92a423040668" (UID: "a8dcf65b-8d86-4217-b1af-92a423040668"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:25:35.357434 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.357415    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef458705-b19a-47d0-a734-c989f57ce834-kube-api-access-scjdx" (OuterVolumeSpecName: "kube-api-access-scjdx") pod "ef458705-b19a-47d0-a734-c989f57ce834" (UID: "ef458705-b19a-47d0-a734-c989f57ce834"). InnerVolumeSpecName "kube-api-access-scjdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:25:35.357514 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.357446    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8dcf65b-8d86-4217-b1af-92a423040668-kube-api-access-z4vzh" (OuterVolumeSpecName: "kube-api-access-z4vzh") pod "a8dcf65b-8d86-4217-b1af-92a423040668" (UID: "a8dcf65b-8d86-4217-b1af-92a423040668"). InnerVolumeSpecName "kube-api-access-z4vzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:25:35.455975 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.455941    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf394dc2-adc9-4eb6-9105-5546a72724a4-plugins-config-volume\") pod \"cf394dc2-adc9-4eb6-9105-5546a72724a4\" (UID: \"cf394dc2-adc9-4eb6-9105-5546a72724a4\") "
Jul 24 23:25:35.456139 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.455996    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mldzc\" (UniqueName: \"kubernetes.io/projected/cf394dc2-adc9-4eb6-9105-5546a72724a4-kube-api-access-mldzc\") pod \"cf394dc2-adc9-4eb6-9105-5546a72724a4\" (UID: \"cf394dc2-adc9-4eb6-9105-5546a72724a4\") "
Jul 24 23:25:35.456218 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.456201    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4vzh\" (UniqueName: \"kubernetes.io/projected/a8dcf65b-8d86-4217-b1af-92a423040668-kube-api-access-z4vzh\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:25:35.456280 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.456224    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scjdx\" (UniqueName: \"kubernetes.io/projected/ef458705-b19a-47d0-a734-c989f57ce834-kube-api-access-scjdx\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:25:35.456280 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.456236    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8dcf65b-8d86-4217-b1af-92a423040668-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:25:35.456350 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.456281    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf394dc2-adc9-4eb6-9105-5546a72724a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf394dc2-adc9-4eb6-9105-5546a72724a4" (UID: "cf394dc2-adc9-4eb6-9105-5546a72724a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:25:35.457932 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.457913    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf394dc2-adc9-4eb6-9105-5546a72724a4-kube-api-access-mldzc" (OuterVolumeSpecName: "kube-api-access-mldzc") pod "cf394dc2-adc9-4eb6-9105-5546a72724a4" (UID: "cf394dc2-adc9-4eb6-9105-5546a72724a4"). InnerVolumeSpecName "kube-api-access-mldzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:25:35.556894 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.556822    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf394dc2-adc9-4eb6-9105-5546a72724a4-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:25:35.556894 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.556842    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mldzc\" (UniqueName: \"kubernetes.io/projected/cf394dc2-adc9-4eb6-9105-5546a72724a4-kube-api-access-mldzc\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:25:35.557766 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.557746    2568 generic.go:356] "Generic (PLEG): container finished" podID="ef458705-b19a-47d0-a734-c989f57ce834" containerID="e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758" exitCode=0
Jul 24 23:25:35.557828 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.557786    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"
Jul 24 23:25:35.557828 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.557819    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4" event={"ID":"ef458705-b19a-47d0-a734-c989f57ce834","Type":"ContainerDied","Data":"e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758"}
Jul 24 23:25:35.557906 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.557869    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4" event={"ID":"ef458705-b19a-47d0-a734-c989f57ce834","Type":"ContainerDied","Data":"8a40b1bbfe883f827286e890b8ef0b4a8b9ecfd1d2bdedfaee4300fb1391de03"}
Jul 24 23:25:35.557906 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.557902    2568 scope.go:117] "RemoveContainer" containerID="e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758"
Jul 24 23:25:35.559111 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.559088    2568 generic.go:356] "Generic (PLEG): container finished" podID="a8dcf65b-8d86-4217-b1af-92a423040668" containerID="d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc" exitCode=0
Jul 24 23:25:35.559198 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.559137    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"
Jul 24 23:25:35.559198 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.559172    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5" event={"ID":"a8dcf65b-8d86-4217-b1af-92a423040668","Type":"ContainerDied","Data":"d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc"}
Jul 24 23:25:35.559330 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.559218    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5" event={"ID":"a8dcf65b-8d86-4217-b1af-92a423040668","Type":"ContainerDied","Data":"db28faa6609ccbe1d81633940b134e36d46ee35f6d184c90fbe69c708720820f"}
Jul 24 23:25:35.560527 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.560509    2568 generic.go:356] "Generic (PLEG): container finished" podID="cf394dc2-adc9-4eb6-9105-5546a72724a4" containerID="f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161" exitCode=0
Jul 24 23:25:35.560636 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.560547    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh" event={"ID":"cf394dc2-adc9-4eb6-9105-5546a72724a4","Type":"ContainerDied","Data":"f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161"}
Jul 24 23:25:35.560636 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.560554    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"
Jul 24 23:25:35.560636 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.560580    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh" event={"ID":"cf394dc2-adc9-4eb6-9105-5546a72724a4","Type":"ContainerDied","Data":"1dce97aa0da5bf78e62cb9eca425763c9e1c137057dc9dc032596111730f343a"}
Jul 24 23:25:35.567286 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.567239    2568 scope.go:117] "RemoveContainer" containerID="e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758"
Jul 24 23:25:35.567580 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:25:35.567563    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758\": container with ID starting with e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758 not found: ID does not exist" containerID="e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758"
Jul 24 23:25:35.567648 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.567588    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758"} err="failed to get container status \"e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758\": rpc error: code = NotFound desc = could not find container \"e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758\": container with ID starting with e8e58fa2530b19bf2ca28a759747ff576dca5b5993ec2fa0fab4f37b35c86758 not found: ID does not exist"
Jul 24 23:25:35.567648 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.567607    2568 scope.go:117] "RemoveContainer" containerID="d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc"
Jul 24 23:25:35.575793 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.575777    2568 scope.go:117] "RemoveContainer" containerID="d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc"
Jul 24 23:25:35.576032 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:25:35.576017    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc\": container with ID starting with d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc not found: ID does not exist" containerID="d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc"
Jul 24 23:25:35.576076 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.576037    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc"} err="failed to get container status \"d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc\": rpc error: code = NotFound desc = could not find container \"d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc\": container with ID starting with d15b107f6205f5bdd4864e4fc62cdf30475bc6fee17b12f4fbc84fbc44bd9cfc not found: ID does not exist"
Jul 24 23:25:35.576076 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.576053    2568 scope.go:117] "RemoveContainer" containerID="f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161"
Jul 24 23:25:35.583130 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.583110    2568 scope.go:117] "RemoveContainer" containerID="f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161"
Jul 24 23:25:35.583434 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:25:35.583415    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161\": container with ID starting with f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161 not found: ID does not exist" containerID="f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161"
Jul 24 23:25:35.583490 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.583439    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161"} err="failed to get container status \"f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161\": rpc error: code = NotFound desc = could not find container \"f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161\": container with ID starting with f45b45136aee5531a33e2c20986ed245a57633c175e710a2aa6e4c999a282161 not found: ID does not exist"
Jul 24 23:25:35.584412 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.584396    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"]
Jul 24 23:25:35.588270 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.588233    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9d44355-67b5b9c9b6-k52b4"]
Jul 24 23:25:35.598056 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.598037    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"]
Jul 24 23:25:35.601597 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.601580    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9d44355-79fd65b7bd-c5dx5"]
Jul 24 23:25:35.612029 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.612011    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"]
Jul 24 23:25:35.617278 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:35.617240    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9d44355-846c65585b-qf8rh"]
Jul 24 23:25:37.257976 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:37.257945    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8dcf65b-8d86-4217-b1af-92a423040668" path="/var/lib/kubelet/pods/a8dcf65b-8d86-4217-b1af-92a423040668/volumes"
Jul 24 23:25:37.258356 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:37.258303    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf394dc2-adc9-4eb6-9105-5546a72724a4" path="/var/lib/kubelet/pods/cf394dc2-adc9-4eb6-9105-5546a72724a4/volumes"
Jul 24 23:25:37.258612 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:25:37.258599    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef458705-b19a-47d0-a734-c989f57ce834" path="/var/lib/kubelet/pods/ef458705-b19a-47d0-a734-c989f57ce834/volumes"
Jul 24 23:26:23.366974 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:26:23.366945    2568 scope.go:117] "RemoveContainer" containerID="2d1a7bd4f8c18e82ba5f87c3115c329b87704524b0bde1746bd2e28fa69348ab"
Jul 24 23:27:31.309781 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.309742    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"]
Jul 24 23:27:31.310401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.310128    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf394dc2-adc9-4eb6-9105-5546a72724a4" containerName="payload-pre-processing"
Jul 24 23:27:31.310401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.310144    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf394dc2-adc9-4eb6-9105-5546a72724a4" containerName="payload-pre-processing"
Jul 24 23:27:31.310401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.310164    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8dcf65b-8d86-4217-b1af-92a423040668" containerName="maas-api"
Jul 24 23:27:31.310401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.310169    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8dcf65b-8d86-4217-b1af-92a423040668" containerName="maas-api"
Jul 24 23:27:31.310401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.310181    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef458705-b19a-47d0-a734-c989f57ce834" containerName="payload-processing"
Jul 24 23:27:31.310401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.310187    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef458705-b19a-47d0-a734-c989f57ce834" containerName="payload-processing"
Jul 24 23:27:31.310401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.310271    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf394dc2-adc9-4eb6-9105-5546a72724a4" containerName="payload-pre-processing"
Jul 24 23:27:31.310401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.310282    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8dcf65b-8d86-4217-b1af-92a423040668" containerName="maas-api"
Jul 24 23:27:31.310401 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.310291    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef458705-b19a-47d0-a734-c989f57ce834" containerName="payload-processing"
Jul 24 23:27:31.313168 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.313149    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:31.316046 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.316021    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-eb73b6-serving-cert\""
Jul 24 23:27:31.326376 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.326348    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"]
Jul 24 23:27:31.406499 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.406458    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr564\" (UniqueName: \"kubernetes.io/projected/b72a4f99-ef54-4a36-b785-da6dae3aecdf-kube-api-access-kr564\") pod \"maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:31.406694 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.406532    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b72a4f99-ef54-4a36-b785-da6dae3aecdf-maas-api-tls\") pod \"maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:31.406694 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.406617    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b72a4f99-ef54-4a36-b785-da6dae3aecdf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:31.507629 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.507590    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr564\" (UniqueName: \"kubernetes.io/projected/b72a4f99-ef54-4a36-b785-da6dae3aecdf-kube-api-access-kr564\") pod \"maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:31.507823 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.507666    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b72a4f99-ef54-4a36-b785-da6dae3aecdf-maas-api-tls\") pod \"maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:31.507823 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.507731    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b72a4f99-ef54-4a36-b785-da6dae3aecdf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:31.507914 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:27:31.507812    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-serving-cert: secret "maas-api-e2e-inf-a-eb73b6-serving-cert" not found
Jul 24 23:27:31.507914 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:27:31.507877    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b72a4f99-ef54-4a36-b785-da6dae3aecdf-maas-api-tls podName:b72a4f99-ef54-4a36-b785-da6dae3aecdf nodeName:}" failed. No retries permitted until 2026-07-24 23:27:32.007860088 +0000 UTC m=+2109.284893610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b72a4f99-ef54-4a36-b785-da6dae3aecdf-maas-api-tls") pod "maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9" (UID: "b72a4f99-ef54-4a36-b785-da6dae3aecdf") : secret "maas-api-e2e-inf-a-eb73b6-serving-cert" not found
Jul 24 23:27:31.508406 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.508390    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b72a4f99-ef54-4a36-b785-da6dae3aecdf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:31.517178 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.517150    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr564\" (UniqueName: \"kubernetes.io/projected/b72a4f99-ef54-4a36-b785-da6dae3aecdf-kube-api-access-kr564\") pod \"maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:31.991326 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.991291    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"]
Jul 24 23:27:31.994852 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.994832    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:27:31.997411 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.997393    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-eb73b6\""
Jul 24 23:27:31.997533 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:31.997401    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-eb73b6-dockercfg-hf9gc\""
Jul 24 23:27:32.004794 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.004770    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"]
Jul 24 23:27:32.013325 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.013297    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b72a4f99-ef54-4a36-b785-da6dae3aecdf-maas-api-tls\") pod \"maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:32.015780 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.015749    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b72a4f99-ef54-4a36-b785-da6dae3aecdf-maas-api-tls\") pod \"maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:32.114788 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.114756    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk8ql\" (UniqueName: \"kubernetes.io/projected/ea99b68a-a920-4338-a0e5-e5939e14ecc1-kube-api-access-zk8ql\") pod \"payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km\" (UID: \"ea99b68a-a920-4338-a0e5-e5939e14ecc1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:27:32.114980 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.114815    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea99b68a-a920-4338-a0e5-e5939e14ecc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km\" (UID: \"ea99b68a-a920-4338-a0e5-e5939e14ecc1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:27:32.215959 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.215927    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk8ql\" (UniqueName: \"kubernetes.io/projected/ea99b68a-a920-4338-a0e5-e5939e14ecc1-kube-api-access-zk8ql\") pod \"payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km\" (UID: \"ea99b68a-a920-4338-a0e5-e5939e14ecc1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:27:32.216122 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.215978    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea99b68a-a920-4338-a0e5-e5939e14ecc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km\" (UID: \"ea99b68a-a920-4338-a0e5-e5939e14ecc1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:27:32.216636 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.216618    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea99b68a-a920-4338-a0e5-e5939e14ecc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km\" (UID: \"ea99b68a-a920-4338-a0e5-e5939e14ecc1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:27:32.223495 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.223474    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:32.224387 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.224367    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk8ql\" (UniqueName: \"kubernetes.io/projected/ea99b68a-a920-4338-a0e5-e5939e14ecc1-kube-api-access-zk8ql\") pod \"payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km\" (UID: \"ea99b68a-a920-4338-a0e5-e5939e14ecc1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:27:32.306564 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.306457    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:27:32.369620 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.369591    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"]
Jul 24 23:27:32.372463 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:27:32.372429    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72a4f99_ef54_4a36_b785_da6dae3aecdf.slice/crio-1d82f1981b389d0739f37cfffd6e1e76530e5e0c67b4202ad89b41c6ee084ca7 WatchSource:0}: Error finding container 1d82f1981b389d0739f37cfffd6e1e76530e5e0c67b4202ad89b41c6ee084ca7: Status 404 returned error can't find the container with id 1d82f1981b389d0739f37cfffd6e1e76530e5e0c67b4202ad89b41c6ee084ca7
Jul 24 23:27:32.446947 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:32.446922    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"]
Jul 24 23:27:32.449280 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:27:32.449227    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea99b68a_a920_4338_a0e5_e5939e14ecc1.slice/crio-ff91b6135f797f4a53a619d01c55bf3eed5c63f4bfdc9e11851fe4c3e1895b8e WatchSource:0}: Error finding container ff91b6135f797f4a53a619d01c55bf3eed5c63f4bfdc9e11851fe4c3e1895b8e: Status 404 returned error can't find the container with id ff91b6135f797f4a53a619d01c55bf3eed5c63f4bfdc9e11851fe4c3e1895b8e
Jul 24 23:27:33.125172 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:33.125062    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km" event={"ID":"ea99b68a-a920-4338-a0e5-e5939e14ecc1","Type":"ContainerStarted","Data":"db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370"}
Jul 24 23:27:33.125172 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:33.125157    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km" event={"ID":"ea99b68a-a920-4338-a0e5-e5939e14ecc1","Type":"ContainerStarted","Data":"ff91b6135f797f4a53a619d01c55bf3eed5c63f4bfdc9e11851fe4c3e1895b8e"}
Jul 24 23:27:33.125422 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:33.125190    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:27:33.126464 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:33.126434    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9" event={"ID":"b72a4f99-ef54-4a36-b785-da6dae3aecdf","Type":"ContainerStarted","Data":"451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6"}
Jul 24 23:27:33.126572 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:33.126484    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9" event={"ID":"b72a4f99-ef54-4a36-b785-da6dae3aecdf","Type":"ContainerStarted","Data":"1d82f1981b389d0739f37cfffd6e1e76530e5e0c67b4202ad89b41c6ee084ca7"}
Jul 24 23:27:33.126572 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:33.126509    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:33.142385 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:33.142339    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km" podStartSLOduration=2.142323632 podStartE2EDuration="2.142323632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:27:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:27:33.140093131 +0000 UTC m=+2110.417126686" watchObservedRunningTime="2026-07-24 23:27:33.142323632 +0000 UTC m=+2110.419357175"
Jul 24 23:27:33.157105 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:33.157045    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9" podStartSLOduration=1.7010493979999999 podStartE2EDuration="2.157027145s" totalImagesPullingTime="455.977747ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:27:33.155327394 +0000 UTC m=+2110.432360947" watchObservedRunningTime="2026-07-24 23:27:33.157027145 +0000 UTC m=+2110.434060690"
Jul 24 23:27:39.137712 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.137687    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:27:39.225295 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.225262    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"]
Jul 24 23:27:39.228820 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.228799    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:39.231349 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.231325    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-0f3852-serving-cert\""
Jul 24 23:27:39.240667 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.240645    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"]
Jul 24 23:27:39.385484 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.385453    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-maas-api-tls\") pod \"maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:39.385657 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.385520    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffpn5\" (UniqueName: \"kubernetes.io/projected/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-kube-api-access-ffpn5\") pod \"maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:39.386228 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.386169    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:39.488074 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.488044    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:39.488218 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.488101    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-maas-api-tls\") pod \"maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:39.488218 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.488129    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffpn5\" (UniqueName: \"kubernetes.io/projected/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-kube-api-access-ffpn5\") pod \"maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:39.488312 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:27:39.488278    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-serving-cert: secret "maas-api-e2e-inf-b-0f3852-serving-cert" not found
Jul 24 23:27:39.488354 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:27:39.488348    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-maas-api-tls podName:2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4 nodeName:}" failed. No retries permitted until 2026-07-24 23:27:39.988328138 +0000 UTC m=+2117.265361663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-maas-api-tls") pod "maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw" (UID: "2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4") : secret "maas-api-e2e-inf-b-0f3852-serving-cert" not found
Jul 24 23:27:39.488658 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.488639    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:39.496587 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.496571    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffpn5\" (UniqueName: \"kubernetes.io/projected/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-kube-api-access-ffpn5\") pod \"maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:39.993199 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.993165    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-maas-api-tls\") pod \"maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:39.995512 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:39.995489    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-maas-api-tls\") pod \"maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:40.054337 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.054309    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"]
Jul 24 23:27:40.057841 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.057825    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:27:40.060884 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.060862    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-0f3852-dockercfg-c4flk\""
Jul 24 23:27:40.060997 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.060880    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-0f3852\""
Jul 24 23:27:40.069083 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.069061    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"]
Jul 24 23:27:40.095206 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.095184    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"]
Jul 24 23:27:40.098461 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.098441    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:27:40.108338 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.108319    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"]
Jul 24 23:27:40.139295 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.139270    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:40.195101 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.195074    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twxng\" (UniqueName: \"kubernetes.io/projected/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-kube-api-access-twxng\") pod \"payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt\" (UID: \"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:27:40.195212 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.195179    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt\" (UID: \"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:27:40.270017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.269995    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"]
Jul 24 23:27:40.271874 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:27:40.271844    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a7ce62f_4dc4_467f_b091_3c4b3b1ca2d4.slice/crio-35ba830bcd4eee4bcadd96bbae79dbe26d93d255f299653da9f46ccbb6f2fea0 WatchSource:0}: Error finding container 35ba830bcd4eee4bcadd96bbae79dbe26d93d255f299653da9f46ccbb6f2fea0: Status 404 returned error can't find the container with id 35ba830bcd4eee4bcadd96bbae79dbe26d93d255f299653da9f46ccbb6f2fea0
Jul 24 23:27:40.296490 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.296467    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twxng\" (UniqueName: \"kubernetes.io/projected/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-kube-api-access-twxng\") pod \"payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt\" (UID: \"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:27:40.296597 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.296521    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29\" (UID: \"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:27:40.296597 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.296541    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpjs8\" (UniqueName: \"kubernetes.io/projected/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-kube-api-access-lpjs8\") pod \"payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29\" (UID: \"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:27:40.296724 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.296604    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt\" (UID: \"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:27:40.297279 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.297234    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt\" (UID: \"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:27:40.304670 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.304646    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twxng\" (UniqueName: \"kubernetes.io/projected/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-kube-api-access-twxng\") pod \"payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt\" (UID: \"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58\") " pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:27:40.368499 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.368472    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:27:40.397013 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.396988    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29\" (UID: \"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:27:40.397132 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.397028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpjs8\" (UniqueName: \"kubernetes.io/projected/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-kube-api-access-lpjs8\") pod \"payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29\" (UID: \"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:27:40.397693 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.397675    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29\" (UID: \"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:27:40.405188 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.405161    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpjs8\" (UniqueName: \"kubernetes.io/projected/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-kube-api-access-lpjs8\") pod \"payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29\" (UID: \"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:27:40.407993 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.407962    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:27:40.510796 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.510758    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"]
Jul 24 23:27:40.514341 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:27:40.514312    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5f3cb1_1ffd_4b7a_897f_2774562a2f58.slice/crio-d80adfe6ccd6d1fb4f2fa1fad8dfc49f6ff3f7d2b5835df0f508a2ad35878477 WatchSource:0}: Error finding container d80adfe6ccd6d1fb4f2fa1fad8dfc49f6ff3f7d2b5835df0f508a2ad35878477: Status 404 returned error can't find the container with id d80adfe6ccd6d1fb4f2fa1fad8dfc49f6ff3f7d2b5835df0f508a2ad35878477
Jul 24 23:27:40.555925 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:40.555894    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"]
Jul 24 23:27:40.558171 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:27:40.558147    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f27ac96_5c78_4a41_b32d_a7cb7d7e0c6a.slice/crio-34cb9c6d06f52a569584d2e571c4e0cb7d80b66b511b457e77e18656ae3c5269 WatchSource:0}: Error finding container 34cb9c6d06f52a569584d2e571c4e0cb7d80b66b511b457e77e18656ae3c5269: Status 404 returned error can't find the container with id 34cb9c6d06f52a569584d2e571c4e0cb7d80b66b511b457e77e18656ae3c5269
Jul 24 23:27:41.167729 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.167693    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt" event={"ID":"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58","Type":"ContainerStarted","Data":"b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f"}
Jul 24 23:27:41.168171 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.167756    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:27:41.168171 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.167780    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt" event={"ID":"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58","Type":"ContainerStarted","Data":"d80adfe6ccd6d1fb4f2fa1fad8dfc49f6ff3f7d2b5835df0f508a2ad35878477"}
Jul 24 23:27:41.169240 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.169215    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw" event={"ID":"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4","Type":"ContainerStarted","Data":"57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27"}
Jul 24 23:27:41.169377 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.169285    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw" event={"ID":"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4","Type":"ContainerStarted","Data":"35ba830bcd4eee4bcadd96bbae79dbe26d93d255f299653da9f46ccbb6f2fea0"}
Jul 24 23:27:41.169377 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.169316    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:41.170602 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.170580    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29" event={"ID":"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a","Type":"ContainerStarted","Data":"2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3"}
Jul 24 23:27:41.170685 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.170621    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29" event={"ID":"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a","Type":"ContainerStarted","Data":"34cb9c6d06f52a569584d2e571c4e0cb7d80b66b511b457e77e18656ae3c5269"}
Jul 24 23:27:41.170685 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.170649    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:27:41.187862 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.187822    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt" podStartSLOduration=1.187809151 podStartE2EDuration="1.187809151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:27:41.186829176 +0000 UTC m=+2118.463862740" watchObservedRunningTime="2026-07-24 23:27:41.187809151 +0000 UTC m=+2118.464842694"
Jul 24 23:27:41.203120 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.203086    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29" podStartSLOduration=1.203075743 podStartE2EDuration="1.203075743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:27:41.201471968 +0000 UTC m=+2118.478505503" watchObservedRunningTime="2026-07-24 23:27:41.203075743 +0000 UTC m=+2118.480109287"
Jul 24 23:27:41.222694 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:41.222655    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw" podStartSLOduration=1.8805661439999999 podStartE2EDuration="2.222644026s" totalImagesPullingTime="342.077882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:27:41.220614809 +0000 UTC m=+2118.497648354" watchObservedRunningTime="2026-07-24 23:27:41.222644026 +0000 UTC m=+2118.499677628"
Jul 24 23:27:44.134128 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:44.134101    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:27:47.181558 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:47.181528    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:27:52.177695 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:52.177652    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:27:52.178168 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:27:52.177760    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:29:35.331761 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.331719    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"]
Jul 24 23:29:35.332286 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.331983    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9" podUID="b72a4f99-ef54-4a36-b785-da6dae3aecdf" containerName="maas-api" containerID="cri-o://451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6" gracePeriod=30
Jul 24 23:29:35.383571 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.383542    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"]
Jul 24 23:29:35.383814 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.383787    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km" podUID="ea99b68a-a920-4338-a0e5-e5939e14ecc1" containerName="payload-pre-processing" containerID="cri-o://db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370" gracePeriod=30
Jul 24 23:29:35.643095 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.643067    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:29:35.645687 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.645667    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:29:35.716210 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.716181    2568 generic.go:356] "Generic (PLEG): container finished" podID="ea99b68a-a920-4338-a0e5-e5939e14ecc1" containerID="db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370" exitCode=0
Jul 24 23:29:35.716407 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.716226    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"
Jul 24 23:29:35.716407 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.716292    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km" event={"ID":"ea99b68a-a920-4338-a0e5-e5939e14ecc1","Type":"ContainerDied","Data":"db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370"}
Jul 24 23:29:35.716407 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.716381    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km" event={"ID":"ea99b68a-a920-4338-a0e5-e5939e14ecc1","Type":"ContainerDied","Data":"ff91b6135f797f4a53a619d01c55bf3eed5c63f4bfdc9e11851fe4c3e1895b8e"}
Jul 24 23:29:35.716581 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.716413    2568 scope.go:117] "RemoveContainer" containerID="db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370"
Jul 24 23:29:35.717510 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.717490    2568 generic.go:356] "Generic (PLEG): container finished" podID="b72a4f99-ef54-4a36-b785-da6dae3aecdf" containerID="451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6" exitCode=0
Jul 24 23:29:35.717580 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.717531    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"
Jul 24 23:29:35.717580 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.717550    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9" event={"ID":"b72a4f99-ef54-4a36-b785-da6dae3aecdf","Type":"ContainerDied","Data":"451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6"}
Jul 24 23:29:35.717650 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.717583    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9" event={"ID":"b72a4f99-ef54-4a36-b785-da6dae3aecdf","Type":"ContainerDied","Data":"1d82f1981b389d0739f37cfffd6e1e76530e5e0c67b4202ad89b41c6ee084ca7"}
Jul 24 23:29:35.727306 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.725258    2568 scope.go:117] "RemoveContainer" containerID="db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370"
Jul 24 23:29:35.727306 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:29:35.725592    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370\": container with ID starting with db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370 not found: ID does not exist" containerID="db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370"
Jul 24 23:29:35.727306 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.725621    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370"} err="failed to get container status \"db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370\": rpc error: code = NotFound desc = could not find container \"db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370\": container with ID starting with db2a0d22f83a84143179401553325f02e78de5d919e7611936af436a6a25b370 not found: ID does not exist"
Jul 24 23:29:35.727306 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.725641    2568 scope.go:117] "RemoveContainer" containerID="451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6"
Jul 24 23:29:35.734846 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.734831    2568 scope.go:117] "RemoveContainer" containerID="451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6"
Jul 24 23:29:35.735064 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:29:35.735047    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6\": container with ID starting with 451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6 not found: ID does not exist" containerID="451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6"
Jul 24 23:29:35.735136 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.735069    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6"} err="failed to get container status \"451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6\": rpc error: code = NotFound desc = could not find container \"451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6\": container with ID starting with 451afed2e312247d1d6a5ca38e60dd13a1487cb8caf69a5bb7f32eec3cc60bc6 not found: ID does not exist"
Jul 24 23:29:35.764829 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.764812    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea99b68a-a920-4338-a0e5-e5939e14ecc1-plugins-config-volume\") pod \"ea99b68a-a920-4338-a0e5-e5939e14ecc1\" (UID: \"ea99b68a-a920-4338-a0e5-e5939e14ecc1\") "
Jul 24 23:29:35.764930 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.764843    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk8ql\" (UniqueName: \"kubernetes.io/projected/ea99b68a-a920-4338-a0e5-e5939e14ecc1-kube-api-access-zk8ql\") pod \"ea99b68a-a920-4338-a0e5-e5939e14ecc1\" (UID: \"ea99b68a-a920-4338-a0e5-e5939e14ecc1\") "
Jul 24 23:29:35.764930 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.764884    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr564\" (UniqueName: \"kubernetes.io/projected/b72a4f99-ef54-4a36-b785-da6dae3aecdf-kube-api-access-kr564\") pod \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") "
Jul 24 23:29:35.765046 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.764986    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b72a4f99-ef54-4a36-b785-da6dae3aecdf-openshift-service-ca\") pod \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") "
Jul 24 23:29:35.765046 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.765018    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b72a4f99-ef54-4a36-b785-da6dae3aecdf-maas-api-tls\") pod \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\" (UID: \"b72a4f99-ef54-4a36-b785-da6dae3aecdf\") "
Jul 24 23:29:35.765187 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.765165    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea99b68a-a920-4338-a0e5-e5939e14ecc1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea99b68a-a920-4338-a0e5-e5939e14ecc1" (UID: "ea99b68a-a920-4338-a0e5-e5939e14ecc1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:29:35.765357 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.765338    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72a4f99-ef54-4a36-b785-da6dae3aecdf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b72a4f99-ef54-4a36-b785-da6dae3aecdf" (UID: "b72a4f99-ef54-4a36-b785-da6dae3aecdf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:29:35.765519 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.765501    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b72a4f99-ef54-4a36-b785-da6dae3aecdf-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:35.765576 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.765521    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea99b68a-a920-4338-a0e5-e5939e14ecc1-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:35.766896 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.766877    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72a4f99-ef54-4a36-b785-da6dae3aecdf-kube-api-access-kr564" (OuterVolumeSpecName: "kube-api-access-kr564") pod "b72a4f99-ef54-4a36-b785-da6dae3aecdf" (UID: "b72a4f99-ef54-4a36-b785-da6dae3aecdf"). InnerVolumeSpecName "kube-api-access-kr564". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:29:35.766971 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.766890    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72a4f99-ef54-4a36-b785-da6dae3aecdf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b72a4f99-ef54-4a36-b785-da6dae3aecdf" (UID: "b72a4f99-ef54-4a36-b785-da6dae3aecdf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:29:35.766971 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.766901    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea99b68a-a920-4338-a0e5-e5939e14ecc1-kube-api-access-zk8ql" (OuterVolumeSpecName: "kube-api-access-zk8ql") pod "ea99b68a-a920-4338-a0e5-e5939e14ecc1" (UID: "ea99b68a-a920-4338-a0e5-e5939e14ecc1"). InnerVolumeSpecName "kube-api-access-zk8ql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:29:35.866885 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.866833    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b72a4f99-ef54-4a36-b785-da6dae3aecdf-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:35.866885 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.866853    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk8ql\" (UniqueName: \"kubernetes.io/projected/ea99b68a-a920-4338-a0e5-e5939e14ecc1-kube-api-access-zk8ql\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:35.866885 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:35.866862    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr564\" (UniqueName: \"kubernetes.io/projected/b72a4f99-ef54-4a36-b785-da6dae3aecdf-kube-api-access-kr564\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:36.041989 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:36.041964    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"]
Jul 24 23:29:36.044057 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:36.044035    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-eb73b6-f7fb6467d-tn7d9"]
Jul 24 23:29:36.054661 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:36.054638    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"]
Jul 24 23:29:36.063557 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:36.063536    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-eb73b6-78d55dbcb8-c59km"]
Jul 24 23:29:37.257663 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:37.257632    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72a4f99-ef54-4a36-b785-da6dae3aecdf" path="/var/lib/kubelet/pods/b72a4f99-ef54-4a36-b785-da6dae3aecdf/volumes"
Jul 24 23:29:37.258022 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:37.257976    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea99b68a-a920-4338-a0e5-e5939e14ecc1" path="/var/lib/kubelet/pods/ea99b68a-a920-4338-a0e5-e5939e14ecc1/volumes"
Jul 24 23:29:57.372904 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.372874    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"]
Jul 24 23:29:57.373360 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.373079    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw" podUID="2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4" containerName="maas-api" containerID="cri-o://57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27" gracePeriod=30
Jul 24 23:29:57.423937 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.423903    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"]
Jul 24 23:29:57.424164 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.424138    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt" podUID="ea5f3cb1-1ffd-4b7a-897f-2774562a2f58" containerName="payload-processing" containerID="cri-o://b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f" gracePeriod=30
Jul 24 23:29:57.428017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.427923    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"]
Jul 24 23:29:57.428206 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.428132    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29" podUID="8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a" containerName="payload-pre-processing" containerID="cri-o://2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3" gracePeriod=30
Jul 24 23:29:57.700688 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.700669    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:29:57.702813 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.702796    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:29:57.723049 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.723025    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:29:57.767298 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.767271    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twxng\" (UniqueName: \"kubernetes.io/projected/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-kube-api-access-twxng\") pod \"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58\" (UID: \"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58\") "
Jul 24 23:29:57.767434 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.767313    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffpn5\" (UniqueName: \"kubernetes.io/projected/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-kube-api-access-ffpn5\") pod \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") "
Jul 24 23:29:57.767434 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.767347    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-plugins-config-volume\") pod \"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58\" (UID: \"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58\") "
Jul 24 23:29:57.767434 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.767382    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-openshift-service-ca\") pod \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") "
Jul 24 23:29:57.767434 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.767409    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-maas-api-tls\") pod \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\" (UID: \"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4\") "
Jul 24 23:29:57.767648 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.767457    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpjs8\" (UniqueName: \"kubernetes.io/projected/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-kube-api-access-lpjs8\") pod \"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a\" (UID: \"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a\") "
Jul 24 23:29:57.767648 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.767482    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-plugins-config-volume\") pod \"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a\" (UID: \"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a\") "
Jul 24 23:29:57.767808 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.767740    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4" (UID: "2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:29:57.767808 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.767772    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea5f3cb1-1ffd-4b7a-897f-2774562a2f58" (UID: "ea5f3cb1-1ffd-4b7a-897f-2774562a2f58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:29:57.768012 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.767986    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:57.768105 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.768020    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:57.768105 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.768064    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a" (UID: "8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:29:57.769721 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.769679    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-kube-api-access-lpjs8" (OuterVolumeSpecName: "kube-api-access-lpjs8") pod "8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a" (UID: "8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a"). InnerVolumeSpecName "kube-api-access-lpjs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:29:57.769808 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.769785    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4" (UID: "2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:29:57.769847 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.769803    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-kube-api-access-twxng" (OuterVolumeSpecName: "kube-api-access-twxng") pod "ea5f3cb1-1ffd-4b7a-897f-2774562a2f58" (UID: "ea5f3cb1-1ffd-4b7a-897f-2774562a2f58"). InnerVolumeSpecName "kube-api-access-twxng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:29:57.769847 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.769828    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-kube-api-access-ffpn5" (OuterVolumeSpecName: "kube-api-access-ffpn5") pod "2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4" (UID: "2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4"). InnerVolumeSpecName "kube-api-access-ffpn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:29:57.827523 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.827501    2568 generic.go:356] "Generic (PLEG): container finished" podID="8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a" containerID="2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3" exitCode=0
Jul 24 23:29:57.827615 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.827542    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"
Jul 24 23:29:57.827615 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.827583    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29" event={"ID":"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a","Type":"ContainerDied","Data":"2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3"}
Jul 24 23:29:57.827711 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.827629    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29" event={"ID":"8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a","Type":"ContainerDied","Data":"34cb9c6d06f52a569584d2e571c4e0cb7d80b66b511b457e77e18656ae3c5269"}
Jul 24 23:29:57.827711 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.827651    2568 scope.go:117] "RemoveContainer" containerID="2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3"
Jul 24 23:29:57.828862 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.828837    2568 generic.go:356] "Generic (PLEG): container finished" podID="ea5f3cb1-1ffd-4b7a-897f-2774562a2f58" containerID="b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f" exitCode=0
Jul 24 23:29:57.828957 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.828876    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"
Jul 24 23:29:57.828957 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.828929    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt" event={"ID":"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58","Type":"ContainerDied","Data":"b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f"}
Jul 24 23:29:57.829060 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.828980    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt" event={"ID":"ea5f3cb1-1ffd-4b7a-897f-2774562a2f58","Type":"ContainerDied","Data":"d80adfe6ccd6d1fb4f2fa1fad8dfc49f6ff3f7d2b5835df0f508a2ad35878477"}
Jul 24 23:29:57.830417 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.830392    2568 generic.go:356] "Generic (PLEG): container finished" podID="2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4" containerID="57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27" exitCode=0
Jul 24 23:29:57.830522 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.830447    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"
Jul 24 23:29:57.830522 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.830444    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw" event={"ID":"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4","Type":"ContainerDied","Data":"57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27"}
Jul 24 23:29:57.830522 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.830502    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw" event={"ID":"2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4","Type":"ContainerDied","Data":"35ba830bcd4eee4bcadd96bbae79dbe26d93d255f299653da9f46ccbb6f2fea0"}
Jul 24 23:29:57.836447 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.836427    2568 scope.go:117] "RemoveContainer" containerID="2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3"
Jul 24 23:29:57.836698 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:29:57.836676    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3\": container with ID starting with 2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3 not found: ID does not exist" containerID="2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3"
Jul 24 23:29:57.836794 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.836703    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3"} err="failed to get container status \"2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3\": rpc error: code = NotFound desc = could not find container \"2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3\": container with ID starting with 2470c09fe0c568082f684f6a6945c9d32ba7158763f49e7c8b958e23cb67ffa3 not found: ID does not exist"
Jul 24 23:29:57.836794 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.836723    2568 scope.go:117] "RemoveContainer" containerID="b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f"
Jul 24 23:29:57.844703 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.844688    2568 scope.go:117] "RemoveContainer" containerID="b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f"
Jul 24 23:29:57.844914 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:29:57.844897    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f\": container with ID starting with b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f not found: ID does not exist" containerID="b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f"
Jul 24 23:29:57.844981 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.844923    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f"} err="failed to get container status \"b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f\": rpc error: code = NotFound desc = could not find container \"b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f\": container with ID starting with b1d5da18a1b8843e2bd0b7b78b31a473fe18ff1cd740dbff9e9402538b4b5b8f not found: ID does not exist"
Jul 24 23:29:57.844981 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.844945    2568 scope.go:117] "RemoveContainer" containerID="57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27"
Jul 24 23:29:57.852233 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.852217    2568 scope.go:117] "RemoveContainer" containerID="57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27"
Jul 24 23:29:57.852529 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:29:57.852513    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27\": container with ID starting with 57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27 not found: ID does not exist" containerID="57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27"
Jul 24 23:29:57.852584 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.852536    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27"} err="failed to get container status \"57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27\": rpc error: code = NotFound desc = could not find container \"57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27\": container with ID starting with 57d6101531753fb008b1d2a467d620a501653105aede9c76d2a86d80a8615b27 not found: ID does not exist"
Jul 24 23:29:57.853598 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.853580    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"]
Jul 24 23:29:57.856918 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.856899    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-0f3852-59756d9d48-zf5xt"]
Jul 24 23:29:57.866520 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.866502    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"]
Jul 24 23:29:57.868470 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.868449    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twxng\" (UniqueName: \"kubernetes.io/projected/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58-kube-api-access-twxng\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:57.868555 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.868476    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffpn5\" (UniqueName: \"kubernetes.io/projected/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-kube-api-access-ffpn5\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:57.868555 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.868490    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:57.868555 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.868502    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpjs8\" (UniqueName: \"kubernetes.io/projected/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-kube-api-access-lpjs8\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:57.868555 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.868514    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:29:57.869955 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.869939    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-0f3852-85c6bbfc6-vjf29"]
Jul 24 23:29:57.879426 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.879382    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"]
Jul 24 23:29:57.884942 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:57.884925    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0f3852-54d7dfc46b-sfkpw"]
Jul 24 23:29:59.257784 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:59.257750    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4" path="/var/lib/kubelet/pods/2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4/volumes"
Jul 24 23:29:59.258152 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:59.258101    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a" path="/var/lib/kubelet/pods/8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a/volumes"
Jul 24 23:29:59.258440 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:29:59.258426    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea5f3cb1-1ffd-4b7a-897f-2774562a2f58" path="/var/lib/kubelet/pods/ea5f3cb1-1ffd-4b7a-897f-2774562a2f58/volumes"
Jul 24 23:30:00.147052 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147018    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s"]
Jul 24 23:30:00.147402 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147390    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72a4f99-ef54-4a36-b785-da6dae3aecdf" containerName="maas-api"
Jul 24 23:30:00.147460 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147403    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72a4f99-ef54-4a36-b785-da6dae3aecdf" containerName="maas-api"
Jul 24 23:30:00.147460 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147414    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea5f3cb1-1ffd-4b7a-897f-2774562a2f58" containerName="payload-processing"
Jul 24 23:30:00.147460 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147420    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea5f3cb1-1ffd-4b7a-897f-2774562a2f58" containerName="payload-processing"
Jul 24 23:30:00.147460 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147433    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4" containerName="maas-api"
Jul 24 23:30:00.147460 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147438    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4" containerName="maas-api"
Jul 24 23:30:00.147460 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147445    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea99b68a-a920-4338-a0e5-e5939e14ecc1" containerName="payload-pre-processing"
Jul 24 23:30:00.147460 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147451    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea99b68a-a920-4338-a0e5-e5939e14ecc1" containerName="payload-pre-processing"
Jul 24 23:30:00.147460 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147461    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a" containerName="payload-pre-processing"
Jul 24 23:30:00.147710 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147466    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a" containerName="payload-pre-processing"
Jul 24 23:30:00.147710 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147518    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b72a4f99-ef54-4a36-b785-da6dae3aecdf" containerName="maas-api"
Jul 24 23:30:00.147710 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147527    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f27ac96-5c78-4a41-b32d-a7cb7d7e0c6a" containerName="payload-pre-processing"
Jul 24 23:30:00.147710 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147533    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a7ce62f-4dc4-467f-b091-3c4b3b1ca2d4" containerName="maas-api"
Jul 24 23:30:00.147710 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147539    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea5f3cb1-1ffd-4b7a-897f-2774562a2f58" containerName="payload-processing"
Jul 24 23:30:00.147710 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.147544    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea99b68a-a920-4338-a0e5-e5939e14ecc1" containerName="payload-pre-processing"
Jul 24 23:30:00.150813 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.150798    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s"
Jul 24 23:30:00.153649 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.153622    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h9zqg\""
Jul 24 23:30:00.159412 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.159387    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s"]
Jul 24 23:30:00.161003 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.160979    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s"
Jul 24 23:30:00.290353 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.290324    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s"]
Jul 24 23:30:00.292592 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:30:00.292564    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c592212_e1f3_4a70_9ad6_d15da3e01353.slice/crio-7494e0e4d691ca3ad08f684793a84d16e181d5ce7be73de82f7ec320f9088ee2 WatchSource:0}: Error finding container 7494e0e4d691ca3ad08f684793a84d16e181d5ce7be73de82f7ec320f9088ee2: Status 404 returned error can't find the container with id 7494e0e4d691ca3ad08f684793a84d16e181d5ce7be73de82f7ec320f9088ee2
Jul 24 23:30:00.848910 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.848831    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s" event={"ID":"0c592212-e1f3-4a70-9ad6-d15da3e01353","Type":"ContainerStarted","Data":"f7d20dbc0f76c5c8662633b75c65804f000170124a8889d30e5318ca08c1f205"}
Jul 24 23:30:00.848910 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.848882    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s" event={"ID":"0c592212-e1f3-4a70-9ad6-d15da3e01353","Type":"ContainerStarted","Data":"7494e0e4d691ca3ad08f684793a84d16e181d5ce7be73de82f7ec320f9088ee2"}
Jul 24 23:30:00.863784 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:00.863734    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s" podStartSLOduration=0.863721554 podStartE2EDuration="863.721554ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:00.862805681 +0000 UTC m=+2258.139839245" watchObservedRunningTime="2026-07-24 23:30:00.863721554 +0000 UTC m=+2258.140755115"
Jul 24 23:30:01.854454 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:01.854421    2568 generic.go:356] "Generic (PLEG): container finished" podID="0c592212-e1f3-4a70-9ad6-d15da3e01353" containerID="f7d20dbc0f76c5c8662633b75c65804f000170124a8889d30e5318ca08c1f205" exitCode=0
Jul 24 23:30:01.854881 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:01.854489    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s" event={"ID":"0c592212-e1f3-4a70-9ad6-d15da3e01353","Type":"ContainerDied","Data":"f7d20dbc0f76c5c8662633b75c65804f000170124a8889d30e5318ca08c1f205"}
Jul 24 23:30:02.982241 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:02.982216    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s"
Jul 24 23:30:03.863983 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:03.863956    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s"
Jul 24 23:30:03.864149 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:03.863985    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748930-jj68s" event={"ID":"0c592212-e1f3-4a70-9ad6-d15da3e01353","Type":"ContainerDied","Data":"7494e0e4d691ca3ad08f684793a84d16e181d5ce7be73de82f7ec320f9088ee2"}
Jul 24 23:30:03.864149 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:03.864028    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7494e0e4d691ca3ad08f684793a84d16e181d5ce7be73de82f7ec320f9088ee2"
Jul 24 23:30:11.920492 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:11.920457    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"]
Jul 24 23:30:11.921079 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:11.920796    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c592212-e1f3-4a70-9ad6-d15da3e01353" containerName="cleanup"
Jul 24 23:30:11.921079 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:11.920806    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c592212-e1f3-4a70-9ad6-d15da3e01353" containerName="cleanup"
Jul 24 23:30:11.921079 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:11.920879    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c592212-e1f3-4a70-9ad6-d15da3e01353" containerName="cleanup"
Jul 24 23:30:11.923921 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:11.923901    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:11.926375 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:11.926351    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-5d3442-serving-cert\""
Jul 24 23:30:11.933929 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:11.933896    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"]
Jul 24 23:30:11.991734 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:11.991711    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0968b08e-b107-49d7-bb8a-5bc99eeba885-maas-api-tls\") pod \"maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:11.991860 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:11.991780    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9qw5\" (UniqueName: \"kubernetes.io/projected/0968b08e-b107-49d7-bb8a-5bc99eeba885-kube-api-access-r9qw5\") pod \"maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:11.991921 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:11.991905    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0968b08e-b107-49d7-bb8a-5bc99eeba885-openshift-service-ca\") pod \"maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:12.093300 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.093241    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0968b08e-b107-49d7-bb8a-5bc99eeba885-openshift-service-ca\") pod \"maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:12.093493 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.093312    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0968b08e-b107-49d7-bb8a-5bc99eeba885-maas-api-tls\") pod \"maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:12.093493 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.093372    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9qw5\" (UniqueName: \"kubernetes.io/projected/0968b08e-b107-49d7-bb8a-5bc99eeba885-kube-api-access-r9qw5\") pod \"maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:12.093493 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:30:12.093455    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-serving-cert: secret "maas-api-e2e-api-a-5d3442-serving-cert" not found
Jul 24 23:30:12.093653 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:30:12.093560    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0968b08e-b107-49d7-bb8a-5bc99eeba885-maas-api-tls podName:0968b08e-b107-49d7-bb8a-5bc99eeba885 nodeName:}" failed. No retries permitted until 2026-07-24 23:30:12.593543679 +0000 UTC m=+2269.870577200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0968b08e-b107-49d7-bb8a-5bc99eeba885-maas-api-tls") pod "maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz" (UID: "0968b08e-b107-49d7-bb8a-5bc99eeba885") : secret "maas-api-e2e-api-a-5d3442-serving-cert" not found
Jul 24 23:30:12.093857 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.093838    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0968b08e-b107-49d7-bb8a-5bc99eeba885-openshift-service-ca\") pod \"maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:12.102659 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.102642    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9qw5\" (UniqueName: \"kubernetes.io/projected/0968b08e-b107-49d7-bb8a-5bc99eeba885-kube-api-access-r9qw5\") pod \"maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:12.597795 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.597762    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0968b08e-b107-49d7-bb8a-5bc99eeba885-maas-api-tls\") pod \"maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:12.600087 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.600068    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0968b08e-b107-49d7-bb8a-5bc99eeba885-maas-api-tls\") pod \"maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:12.687187 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.687155    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"]
Jul 24 23:30:12.690757 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.690741    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:12.694637 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.694616    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-5d3442\""
Jul 24 23:30:12.713182 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.713159    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"]
Jul 24 23:30:12.772425 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.772399    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"]
Jul 24 23:30:12.775769 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.775754    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:12.779192 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.779177    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-5d3442-dockercfg-z68kk\""
Jul 24 23:30:12.799096 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.799072    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"]
Jul 24 23:30:12.799241 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.799224    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/382b962e-b6db-4030-882a-c13caf0fac63-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q\" (UID: \"382b962e-b6db-4030-882a-c13caf0fac63\") " pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:12.799352 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.799339    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j826w\" (UniqueName: \"kubernetes.io/projected/382b962e-b6db-4030-882a-c13caf0fac63-kube-api-access-j826w\") pod \"payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q\" (UID: \"382b962e-b6db-4030-882a-c13caf0fac63\") " pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:12.834786 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.834757    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:12.900756 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.900651    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n88xz\" (UniqueName: \"kubernetes.io/projected/0bf91a4c-b72c-4476-a33a-7096e62d6c25-kube-api-access-n88xz\") pod \"payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5\" (UID: \"0bf91a4c-b72c-4476-a33a-7096e62d6c25\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:12.900756 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.900697    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/382b962e-b6db-4030-882a-c13caf0fac63-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q\" (UID: \"382b962e-b6db-4030-882a-c13caf0fac63\") " pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:12.900756 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.900729    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bf91a4c-b72c-4476-a33a-7096e62d6c25-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5\" (UID: \"0bf91a4c-b72c-4476-a33a-7096e62d6c25\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:12.901018 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.900815    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j826w\" (UniqueName: \"kubernetes.io/projected/382b962e-b6db-4030-882a-c13caf0fac63-kube-api-access-j826w\") pod \"payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q\" (UID: \"382b962e-b6db-4030-882a-c13caf0fac63\") " pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:12.901397 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.901375    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/382b962e-b6db-4030-882a-c13caf0fac63-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q\" (UID: \"382b962e-b6db-4030-882a-c13caf0fac63\") " pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:12.909935 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.909913    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j826w\" (UniqueName: \"kubernetes.io/projected/382b962e-b6db-4030-882a-c13caf0fac63-kube-api-access-j826w\") pod \"payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q\" (UID: \"382b962e-b6db-4030-882a-c13caf0fac63\") " pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:12.970085 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.970061    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"]
Jul 24 23:30:12.970822 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:30:12.970796    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0968b08e_b107_49d7_bb8a_5bc99eeba885.slice/crio-bda9a22cb36fa548def0d29be53c69d9cb6a34725716c2499cd0932905d279d6 WatchSource:0}: Error finding container bda9a22cb36fa548def0d29be53c69d9cb6a34725716c2499cd0932905d279d6: Status 404 returned error can't find the container with id bda9a22cb36fa548def0d29be53c69d9cb6a34725716c2499cd0932905d279d6
Jul 24 23:30:12.972181 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:12.972165    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:30:13.001140 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.001113    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:13.001442 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.001422    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n88xz\" (UniqueName: \"kubernetes.io/projected/0bf91a4c-b72c-4476-a33a-7096e62d6c25-kube-api-access-n88xz\") pod \"payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5\" (UID: \"0bf91a4c-b72c-4476-a33a-7096e62d6c25\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:13.001531 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.001474    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bf91a4c-b72c-4476-a33a-7096e62d6c25-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5\" (UID: \"0bf91a4c-b72c-4476-a33a-7096e62d6c25\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:13.002094 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.002072    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bf91a4c-b72c-4476-a33a-7096e62d6c25-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5\" (UID: \"0bf91a4c-b72c-4476-a33a-7096e62d6c25\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:13.010408 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.010389    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n88xz\" (UniqueName: \"kubernetes.io/projected/0bf91a4c-b72c-4476-a33a-7096e62d6c25-kube-api-access-n88xz\") pod \"payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5\" (UID: \"0bf91a4c-b72c-4476-a33a-7096e62d6c25\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:13.085606 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.085577    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:13.132762 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.132690    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"]
Jul 24 23:30:13.134313 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:30:13.134290    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod382b962e_b6db_4030_882a_c13caf0fac63.slice/crio-6091c81a93377bc1b9d616576582641fc2e5a61d35fde740d4f1e405437961b5 WatchSource:0}: Error finding container 6091c81a93377bc1b9d616576582641fc2e5a61d35fde740d4f1e405437961b5: Status 404 returned error can't find the container with id 6091c81a93377bc1b9d616576582641fc2e5a61d35fde740d4f1e405437961b5
Jul 24 23:30:13.244494 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.244470    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"]
Jul 24 23:30:13.246640 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:30:13.246611    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bf91a4c_b72c_4476_a33a_7096e62d6c25.slice/crio-6955773618e452ad493fb49c1d54b661abb7bc5a7869b2e6ae26388b653a5cf0 WatchSource:0}: Error finding container 6955773618e452ad493fb49c1d54b661abb7bc5a7869b2e6ae26388b653a5cf0: Status 404 returned error can't find the container with id 6955773618e452ad493fb49c1d54b661abb7bc5a7869b2e6ae26388b653a5cf0
Jul 24 23:30:13.912157 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.912117    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q" event={"ID":"382b962e-b6db-4030-882a-c13caf0fac63","Type":"ContainerStarted","Data":"425e288051633ad3432d60a86e8d5e25edda756635235d910ed9d7a78aa915d6"}
Jul 24 23:30:13.912348 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.912171    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q" event={"ID":"382b962e-b6db-4030-882a-c13caf0fac63","Type":"ContainerStarted","Data":"6091c81a93377bc1b9d616576582641fc2e5a61d35fde740d4f1e405437961b5"}
Jul 24 23:30:13.912348 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.912198    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:13.913703 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.913674    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz" event={"ID":"0968b08e-b107-49d7-bb8a-5bc99eeba885","Type":"ContainerStarted","Data":"e111f7cd143b6ca85a4f6bfaad91a1f2670e5948b501e9f2a7f9d7d5dea9fb1b"}
Jul 24 23:30:13.913811 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.913724    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:13.913811 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.913741    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz" event={"ID":"0968b08e-b107-49d7-bb8a-5bc99eeba885","Type":"ContainerStarted","Data":"bda9a22cb36fa548def0d29be53c69d9cb6a34725716c2499cd0932905d279d6"}
Jul 24 23:30:13.915027 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.915006    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5" event={"ID":"0bf91a4c-b72c-4476-a33a-7096e62d6c25","Type":"ContainerStarted","Data":"65311db655eb26fc3494b3450141836bc1758f834677e2eb9f570c447d491591"}
Jul 24 23:30:13.915121 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.915048    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5" event={"ID":"0bf91a4c-b72c-4476-a33a-7096e62d6c25","Type":"ContainerStarted","Data":"6955773618e452ad493fb49c1d54b661abb7bc5a7869b2e6ae26388b653a5cf0"}
Jul 24 23:30:13.915121 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.915086    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:13.927008 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.926965    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q" podStartSLOduration=1.926949528 podStartE2EDuration="1.926949528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:13.926849766 +0000 UTC m=+2271.203883311" watchObservedRunningTime="2026-07-24 23:30:13.926949528 +0000 UTC m=+2271.203983072"
Jul 24 23:30:13.950002 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.949954    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz" podStartSLOduration=2.575221958 podStartE2EDuration="2.949940489s" totalImagesPullingTime="374.718531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:13.947995053 +0000 UTC m=+2271.225028615" watchObservedRunningTime="2026-07-24 23:30:13.949940489 +0000 UTC m=+2271.226974030"
Jul 24 23:30:13.963668 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:13.963617    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5" podStartSLOduration=1.963603255 podStartE2EDuration="1.963603255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:13.961940159 +0000 UTC m=+2271.238973701" watchObservedRunningTime="2026-07-24 23:30:13.963603255 +0000 UTC m=+2271.240636798"
Jul 24 23:30:19.926078 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:19.926051    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:20.024089 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.024047    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"]
Jul 24 23:30:20.027757 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.027739    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.030499 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.030479    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-183bb6-serving-cert\""
Jul 24 23:30:20.043636 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.043615    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"]
Jul 24 23:30:20.177664 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.177599    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89f4c1e6-9685-4bed-b241-caefe46f9980-maas-api-tls\") pod \"maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.177664 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.177648    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt6xm\" (UniqueName: \"kubernetes.io/projected/89f4c1e6-9685-4bed-b241-caefe46f9980-kube-api-access-zt6xm\") pod \"maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.177812 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.177713    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f4c1e6-9685-4bed-b241-caefe46f9980-openshift-service-ca\") pod \"maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.278806 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.278782    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt6xm\" (UniqueName: \"kubernetes.io/projected/89f4c1e6-9685-4bed-b241-caefe46f9980-kube-api-access-zt6xm\") pod \"maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.278928 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.278812    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f4c1e6-9685-4bed-b241-caefe46f9980-openshift-service-ca\") pod \"maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.278928 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.278871    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89f4c1e6-9685-4bed-b241-caefe46f9980-maas-api-tls\") pod \"maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.279532 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.279514    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f4c1e6-9685-4bed-b241-caefe46f9980-openshift-service-ca\") pod \"maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.281198 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.281178    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89f4c1e6-9685-4bed-b241-caefe46f9980-maas-api-tls\") pod \"maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.294315 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.294293    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt6xm\" (UniqueName: \"kubernetes.io/projected/89f4c1e6-9685-4bed-b241-caefe46f9980-kube-api-access-zt6xm\") pod \"maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.338143 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.338122    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.479304 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.479282    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"]
Jul 24 23:30:20.480986 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:30:20.480960    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89f4c1e6_9685_4bed_b241_caefe46f9980.slice/crio-75fc7f408510437f39f63a8965b89ad3c8eee9dc2dd5f3d472124dba1091cb6a WatchSource:0}: Error finding container 75fc7f408510437f39f63a8965b89ad3c8eee9dc2dd5f3d472124dba1091cb6a: Status 404 returned error can't find the container with id 75fc7f408510437f39f63a8965b89ad3c8eee9dc2dd5f3d472124dba1091cb6a
Jul 24 23:30:20.949818 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.949783    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2" event={"ID":"89f4c1e6-9685-4bed-b241-caefe46f9980","Type":"ContainerStarted","Data":"7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5"}
Jul 24 23:30:20.950275 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.949872    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2" event={"ID":"89f4c1e6-9685-4bed-b241-caefe46f9980","Type":"ContainerStarted","Data":"75fc7f408510437f39f63a8965b89ad3c8eee9dc2dd5f3d472124dba1091cb6a"}
Jul 24 23:30:20.950275 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.949921    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:20.967650 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:20.967583    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2" podStartSLOduration=0.645527594 podStartE2EDuration="967.566054ms" totalImagesPullingTime="322.03846ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:20.967036712 +0000 UTC m=+2278.244070277" watchObservedRunningTime="2026-07-24 23:30:20.967566054 +0000 UTC m=+2278.244599597"
Jul 24 23:30:21.003701 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.003668    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"]
Jul 24 23:30:21.007107 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.007093    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:30:21.009805 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.009781    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-183bb6-dockercfg-qghxm\""
Jul 24 23:30:21.009921 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.009818    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-183bb6\""
Jul 24 23:30:21.017985 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.017965    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"]
Jul 24 23:30:21.087641 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.087553    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee03bc71-8ebe-460d-84ec-1759d1940159-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw\" (UID: \"ee03bc71-8ebe-460d-84ec-1759d1940159\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:30:21.087788 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.087703    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj8db\" (UniqueName: \"kubernetes.io/projected/ee03bc71-8ebe-460d-84ec-1759d1940159-kube-api-access-mj8db\") pod \"payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw\" (UID: \"ee03bc71-8ebe-460d-84ec-1759d1940159\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:30:21.188749 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.188717    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee03bc71-8ebe-460d-84ec-1759d1940159-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw\" (UID: \"ee03bc71-8ebe-460d-84ec-1759d1940159\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:30:21.188935 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.188788    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj8db\" (UniqueName: \"kubernetes.io/projected/ee03bc71-8ebe-460d-84ec-1759d1940159-kube-api-access-mj8db\") pod \"payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw\" (UID: \"ee03bc71-8ebe-460d-84ec-1759d1940159\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:30:21.189443 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.189423    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee03bc71-8ebe-460d-84ec-1759d1940159-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw\" (UID: \"ee03bc71-8ebe-460d-84ec-1759d1940159\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:30:21.198021 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.198003    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj8db\" (UniqueName: \"kubernetes.io/projected/ee03bc71-8ebe-460d-84ec-1759d1940159-kube-api-access-mj8db\") pod \"payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw\" (UID: \"ee03bc71-8ebe-460d-84ec-1759d1940159\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:30:21.317574 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.317540    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:30:21.653527 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.653502    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"]
Jul 24 23:30:21.655207 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:30:21.655174    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee03bc71_8ebe_460d_84ec_1759d1940159.slice/crio-472da1bda7b56ad8df0e5a4d5675a754b664e69d2520df0e20abfd02cc8967eb WatchSource:0}: Error finding container 472da1bda7b56ad8df0e5a4d5675a754b664e69d2520df0e20abfd02cc8967eb: Status 404 returned error can't find the container with id 472da1bda7b56ad8df0e5a4d5675a754b664e69d2520df0e20abfd02cc8967eb
Jul 24 23:30:21.956128 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.956097    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw" event={"ID":"ee03bc71-8ebe-460d-84ec-1759d1940159","Type":"ContainerStarted","Data":"9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d"}
Jul 24 23:30:21.956565 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.956177    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw" event={"ID":"ee03bc71-8ebe-460d-84ec-1759d1940159","Type":"ContainerStarted","Data":"472da1bda7b56ad8df0e5a4d5675a754b664e69d2520df0e20abfd02cc8967eb"}
Jul 24 23:30:21.956565 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.956337    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:30:21.972875 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:21.972835    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw" podStartSLOduration=1.9728214899999998 podStartE2EDuration="1.97282149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:30:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:30:21.970636308 +0000 UTC m=+2279.247669853" watchObservedRunningTime="2026-07-24 23:30:21.97282149 +0000 UTC m=+2279.249855032"
Jul 24 23:30:23.416311 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:23.416286    2568 scope.go:117] "RemoveContainer" containerID="c0f636c45ecb98f80ac79e22ed9b7655d2bf3a4052ef5238c28662d5a78bd9cb"
Jul 24 23:30:24.922907 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:24.922880    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:24.923366 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:24.922950    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:26.961366 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:26.961337    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:30:32.964272 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:32.964218    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:30:47.790362 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:47.790327    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"]
Jul 24 23:30:47.790844 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:47.790578    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz" podUID="0968b08e-b107-49d7-bb8a-5bc99eeba885" containerName="maas-api" containerID="cri-o://e111f7cd143b6ca85a4f6bfaad91a1f2670e5948b501e9f2a7f9d7d5dea9fb1b" gracePeriod=30
Jul 24 23:30:47.852432 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:47.852395    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"]
Jul 24 23:30:47.852731 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:47.852636    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q" podUID="382b962e-b6db-4030-882a-c13caf0fac63" containerName="payload-processing" containerID="cri-o://425e288051633ad3432d60a86e8d5e25edda756635235d910ed9d7a78aa915d6" gracePeriod=30
Jul 24 23:30:47.858799 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:47.858771    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"]
Jul 24 23:30:47.859039 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:47.858994    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5" podUID="0bf91a4c-b72c-4476-a33a-7096e62d6c25" containerName="payload-pre-processing" containerID="cri-o://65311db655eb26fc3494b3450141836bc1758f834677e2eb9f570c447d491591" gracePeriod=30
Jul 24 23:30:48.082045 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.082014    2568 generic.go:356] "Generic (PLEG): container finished" podID="0bf91a4c-b72c-4476-a33a-7096e62d6c25" containerID="65311db655eb26fc3494b3450141836bc1758f834677e2eb9f570c447d491591" exitCode=0
Jul 24 23:30:48.082174 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.082059    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5" event={"ID":"0bf91a4c-b72c-4476-a33a-7096e62d6c25","Type":"ContainerDied","Data":"65311db655eb26fc3494b3450141836bc1758f834677e2eb9f570c447d491591"}
Jul 24 23:30:48.084175 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.084152    2568 generic.go:356] "Generic (PLEG): container finished" podID="382b962e-b6db-4030-882a-c13caf0fac63" containerID="425e288051633ad3432d60a86e8d5e25edda756635235d910ed9d7a78aa915d6" exitCode=0
Jul 24 23:30:48.084348 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.084233    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q" event={"ID":"382b962e-b6db-4030-882a-c13caf0fac63","Type":"ContainerDied","Data":"425e288051633ad3432d60a86e8d5e25edda756635235d910ed9d7a78aa915d6"}
Jul 24 23:30:48.085952 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.085931    2568 generic.go:356] "Generic (PLEG): container finished" podID="0968b08e-b107-49d7-bb8a-5bc99eeba885" containerID="e111f7cd143b6ca85a4f6bfaad91a1f2670e5948b501e9f2a7f9d7d5dea9fb1b" exitCode=0
Jul 24 23:30:48.086068 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.085981    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz" event={"ID":"0968b08e-b107-49d7-bb8a-5bc99eeba885","Type":"ContainerDied","Data":"e111f7cd143b6ca85a4f6bfaad91a1f2670e5948b501e9f2a7f9d7d5dea9fb1b"}
Jul 24 23:30:48.129806 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.129781    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:48.143876 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.143851    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:48.149937 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.149916    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:48.246106 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246074    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bf91a4c-b72c-4476-a33a-7096e62d6c25-plugins-config-volume\") pod \"0bf91a4c-b72c-4476-a33a-7096e62d6c25\" (UID: \"0bf91a4c-b72c-4476-a33a-7096e62d6c25\") "
Jul 24 23:30:48.246325 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246132    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j826w\" (UniqueName: \"kubernetes.io/projected/382b962e-b6db-4030-882a-c13caf0fac63-kube-api-access-j826w\") pod \"382b962e-b6db-4030-882a-c13caf0fac63\" (UID: \"382b962e-b6db-4030-882a-c13caf0fac63\") "
Jul 24 23:30:48.246325 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246157    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9qw5\" (UniqueName: \"kubernetes.io/projected/0968b08e-b107-49d7-bb8a-5bc99eeba885-kube-api-access-r9qw5\") pod \"0968b08e-b107-49d7-bb8a-5bc99eeba885\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") "
Jul 24 23:30:48.246325 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246178    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n88xz\" (UniqueName: \"kubernetes.io/projected/0bf91a4c-b72c-4476-a33a-7096e62d6c25-kube-api-access-n88xz\") pod \"0bf91a4c-b72c-4476-a33a-7096e62d6c25\" (UID: \"0bf91a4c-b72c-4476-a33a-7096e62d6c25\") "
Jul 24 23:30:48.246325 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246193    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/382b962e-b6db-4030-882a-c13caf0fac63-plugins-config-volume\") pod \"382b962e-b6db-4030-882a-c13caf0fac63\" (UID: \"382b962e-b6db-4030-882a-c13caf0fac63\") "
Jul 24 23:30:48.246325 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246215    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0968b08e-b107-49d7-bb8a-5bc99eeba885-maas-api-tls\") pod \"0968b08e-b107-49d7-bb8a-5bc99eeba885\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") "
Jul 24 23:30:48.246604 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246405    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0968b08e-b107-49d7-bb8a-5bc99eeba885-openshift-service-ca\") pod \"0968b08e-b107-49d7-bb8a-5bc99eeba885\" (UID: \"0968b08e-b107-49d7-bb8a-5bc99eeba885\") "
Jul 24 23:30:48.246604 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246571    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bf91a4c-b72c-4476-a33a-7096e62d6c25-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0bf91a4c-b72c-4476-a33a-7096e62d6c25" (UID: "0bf91a4c-b72c-4476-a33a-7096e62d6c25"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:30:48.246700 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246620    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/382b962e-b6db-4030-882a-c13caf0fac63-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "382b962e-b6db-4030-882a-c13caf0fac63" (UID: "382b962e-b6db-4030-882a-c13caf0fac63"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:30:48.246889 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246867    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0968b08e-b107-49d7-bb8a-5bc99eeba885-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0968b08e-b107-49d7-bb8a-5bc99eeba885" (UID: "0968b08e-b107-49d7-bb8a-5bc99eeba885"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:30:48.246889 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246879    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/382b962e-b6db-4030-882a-c13caf0fac63-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:30:48.247012 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.246901    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bf91a4c-b72c-4476-a33a-7096e62d6c25-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:30:48.248871 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.248854    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0968b08e-b107-49d7-bb8a-5bc99eeba885-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0968b08e-b107-49d7-bb8a-5bc99eeba885" (UID: "0968b08e-b107-49d7-bb8a-5bc99eeba885"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:30:48.248871 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.248860    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0968b08e-b107-49d7-bb8a-5bc99eeba885-kube-api-access-r9qw5" (OuterVolumeSpecName: "kube-api-access-r9qw5") pod "0968b08e-b107-49d7-bb8a-5bc99eeba885" (UID: "0968b08e-b107-49d7-bb8a-5bc99eeba885"). InnerVolumeSpecName "kube-api-access-r9qw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:30:48.248986 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.248863    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/382b962e-b6db-4030-882a-c13caf0fac63-kube-api-access-j826w" (OuterVolumeSpecName: "kube-api-access-j826w") pod "382b962e-b6db-4030-882a-c13caf0fac63" (UID: "382b962e-b6db-4030-882a-c13caf0fac63"). InnerVolumeSpecName "kube-api-access-j826w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:30:48.248986 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.248883    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bf91a4c-b72c-4476-a33a-7096e62d6c25-kube-api-access-n88xz" (OuterVolumeSpecName: "kube-api-access-n88xz") pod "0bf91a4c-b72c-4476-a33a-7096e62d6c25" (UID: "0bf91a4c-b72c-4476-a33a-7096e62d6c25"). InnerVolumeSpecName "kube-api-access-n88xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:30:48.348320 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.348211    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0968b08e-b107-49d7-bb8a-5bc99eeba885-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:30:48.348320 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.348272    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j826w\" (UniqueName: \"kubernetes.io/projected/382b962e-b6db-4030-882a-c13caf0fac63-kube-api-access-j826w\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:30:48.348320 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.348286    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9qw5\" (UniqueName: \"kubernetes.io/projected/0968b08e-b107-49d7-bb8a-5bc99eeba885-kube-api-access-r9qw5\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:30:48.348320 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.348299    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n88xz\" (UniqueName: \"kubernetes.io/projected/0bf91a4c-b72c-4476-a33a-7096e62d6c25-kube-api-access-n88xz\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:30:48.348320 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:48.348312    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0968b08e-b107-49d7-bb8a-5bc99eeba885-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:30:49.091843 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.091766    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"
Jul 24 23:30:49.092270 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.091766    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q" event={"ID":"382b962e-b6db-4030-882a-c13caf0fac63","Type":"ContainerDied","Data":"6091c81a93377bc1b9d616576582641fc2e5a61d35fde740d4f1e405437961b5"}
Jul 24 23:30:49.092270 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.091895    2568 scope.go:117] "RemoveContainer" containerID="425e288051633ad3432d60a86e8d5e25edda756635235d910ed9d7a78aa915d6"
Jul 24 23:30:49.093186 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.093176    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"
Jul 24 23:30:49.093267 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.093218    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz" event={"ID":"0968b08e-b107-49d7-bb8a-5bc99eeba885","Type":"ContainerDied","Data":"bda9a22cb36fa548def0d29be53c69d9cb6a34725716c2499cd0932905d279d6"}
Jul 24 23:30:49.094699 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.094679    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"
Jul 24 23:30:49.094799 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.094696    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5" event={"ID":"0bf91a4c-b72c-4476-a33a-7096e62d6c25","Type":"ContainerDied","Data":"6955773618e452ad493fb49c1d54b661abb7bc5a7869b2e6ae26388b653a5cf0"}
Jul 24 23:30:49.102474 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.102455    2568 scope.go:117] "RemoveContainer" containerID="e111f7cd143b6ca85a4f6bfaad91a1f2670e5948b501e9f2a7f9d7d5dea9fb1b"
Jul 24 23:30:49.110230 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.110214    2568 scope.go:117] "RemoveContainer" containerID="65311db655eb26fc3494b3450141836bc1758f834677e2eb9f570c447d491591"
Jul 24 23:30:49.143448 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.143428    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"]
Jul 24 23:30:49.152615 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.152592    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d3442-59bf449d6b-5k6tz"]
Jul 24 23:30:49.187025 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.186997    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"]
Jul 24 23:30:49.193257 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.193227    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5d3442-689f79ccfb-krx2q"]
Jul 24 23:30:49.224408 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.224366    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"]
Jul 24 23:30:49.236496 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.236469    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5d3442-5db69d9784-d5xm5"]
Jul 24 23:30:49.257515 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.257485    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0968b08e-b107-49d7-bb8a-5bc99eeba885" path="/var/lib/kubelet/pods/0968b08e-b107-49d7-bb8a-5bc99eeba885/volumes"
Jul 24 23:30:49.257856 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.257841    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bf91a4c-b72c-4476-a33a-7096e62d6c25" path="/var/lib/kubelet/pods/0bf91a4c-b72c-4476-a33a-7096e62d6c25/volumes"
Jul 24 23:30:49.258193 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:30:49.258180    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="382b962e-b6db-4030-882a-c13caf0fac63" path="/var/lib/kubelet/pods/382b962e-b6db-4030-882a-c13caf0fac63/volumes"
Jul 24 23:31:15.156801 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.156709    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"]
Jul 24 23:31:15.157198 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.156975    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2" podUID="89f4c1e6-9685-4bed-b241-caefe46f9980" containerName="maas-api" containerID="cri-o://7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5" gracePeriod=30
Jul 24 23:31:15.224605 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.224559    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"]
Jul 24 23:31:15.224854 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.224827    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw" podUID="ee03bc71-8ebe-460d-84ec-1759d1940159" containerName="payload-pre-processing" containerID="cri-o://9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d" gracePeriod=30
Jul 24 23:31:15.464373 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.464348    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:31:15.490709 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.490688    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:31:15.587201 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.587171    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89f4c1e6-9685-4bed-b241-caefe46f9980-maas-api-tls\") pod \"89f4c1e6-9685-4bed-b241-caefe46f9980\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") "
Jul 24 23:31:15.587376 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.587240    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt6xm\" (UniqueName: \"kubernetes.io/projected/89f4c1e6-9685-4bed-b241-caefe46f9980-kube-api-access-zt6xm\") pod \"89f4c1e6-9685-4bed-b241-caefe46f9980\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") "
Jul 24 23:31:15.587376 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.587325    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f4c1e6-9685-4bed-b241-caefe46f9980-openshift-service-ca\") pod \"89f4c1e6-9685-4bed-b241-caefe46f9980\" (UID: \"89f4c1e6-9685-4bed-b241-caefe46f9980\") "
Jul 24 23:31:15.587688 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.587651    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89f4c1e6-9685-4bed-b241-caefe46f9980-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "89f4c1e6-9685-4bed-b241-caefe46f9980" (UID: "89f4c1e6-9685-4bed-b241-caefe46f9980"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:15.589277 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.589242    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89f4c1e6-9685-4bed-b241-caefe46f9980-kube-api-access-zt6xm" (OuterVolumeSpecName: "kube-api-access-zt6xm") pod "89f4c1e6-9685-4bed-b241-caefe46f9980" (UID: "89f4c1e6-9685-4bed-b241-caefe46f9980"). InnerVolumeSpecName "kube-api-access-zt6xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:31:15.589365 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.589280    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89f4c1e6-9685-4bed-b241-caefe46f9980-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "89f4c1e6-9685-4bed-b241-caefe46f9980" (UID: "89f4c1e6-9685-4bed-b241-caefe46f9980"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:31:15.688694 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.688668    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee03bc71-8ebe-460d-84ec-1759d1940159-plugins-config-volume\") pod \"ee03bc71-8ebe-460d-84ec-1759d1940159\" (UID: \"ee03bc71-8ebe-460d-84ec-1759d1940159\") "
Jul 24 23:31:15.688834 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.688742    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj8db\" (UniqueName: \"kubernetes.io/projected/ee03bc71-8ebe-460d-84ec-1759d1940159-kube-api-access-mj8db\") pod \"ee03bc71-8ebe-460d-84ec-1759d1940159\" (UID: \"ee03bc71-8ebe-460d-84ec-1759d1940159\") "
Jul 24 23:31:15.689041 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.689020    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee03bc71-8ebe-460d-84ec-1759d1940159-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee03bc71-8ebe-460d-84ec-1759d1940159" (UID: "ee03bc71-8ebe-460d-84ec-1759d1940159"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:31:15.689115 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.689102    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89f4c1e6-9685-4bed-b241-caefe46f9980-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:31:15.689150 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.689118    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89f4c1e6-9685-4bed-b241-caefe46f9980-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:31:15.689150 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.689127    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee03bc71-8ebe-460d-84ec-1759d1940159-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:31:15.689215 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.689154    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt6xm\" (UniqueName: \"kubernetes.io/projected/89f4c1e6-9685-4bed-b241-caefe46f9980-kube-api-access-zt6xm\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:31:15.690683 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.690664    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee03bc71-8ebe-460d-84ec-1759d1940159-kube-api-access-mj8db" (OuterVolumeSpecName: "kube-api-access-mj8db") pod "ee03bc71-8ebe-460d-84ec-1759d1940159" (UID: "ee03bc71-8ebe-460d-84ec-1759d1940159"). InnerVolumeSpecName "kube-api-access-mj8db". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:31:15.790295 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:15.790237    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj8db\" (UniqueName: \"kubernetes.io/projected/ee03bc71-8ebe-460d-84ec-1759d1940159-kube-api-access-mj8db\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:31:16.229225 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.229187    2568 generic.go:356] "Generic (PLEG): container finished" podID="89f4c1e6-9685-4bed-b241-caefe46f9980" containerID="7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5" exitCode=0
Jul 24 23:31:16.229225 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.229229    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"
Jul 24 23:31:16.229729 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.229275    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2" event={"ID":"89f4c1e6-9685-4bed-b241-caefe46f9980","Type":"ContainerDied","Data":"7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5"}
Jul 24 23:31:16.229729 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.229333    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2" event={"ID":"89f4c1e6-9685-4bed-b241-caefe46f9980","Type":"ContainerDied","Data":"75fc7f408510437f39f63a8965b89ad3c8eee9dc2dd5f3d472124dba1091cb6a"}
Jul 24 23:31:16.229729 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.229364    2568 scope.go:117] "RemoveContainer" containerID="7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5"
Jul 24 23:31:16.230659 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.230595    2568 generic.go:356] "Generic (PLEG): container finished" podID="ee03bc71-8ebe-460d-84ec-1759d1940159" containerID="9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d" exitCode=0
Jul 24 23:31:16.230659 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.230634    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw" event={"ID":"ee03bc71-8ebe-460d-84ec-1759d1940159","Type":"ContainerDied","Data":"9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d"}
Jul 24 23:31:16.230659 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.230639    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"
Jul 24 23:31:16.230856 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.230667    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw" event={"ID":"ee03bc71-8ebe-460d-84ec-1759d1940159","Type":"ContainerDied","Data":"472da1bda7b56ad8df0e5a4d5675a754b664e69d2520df0e20abfd02cc8967eb"}
Jul 24 23:31:16.239183 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.239158    2568 scope.go:117] "RemoveContainer" containerID="7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5"
Jul 24 23:31:16.239516 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:31:16.239497    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5\": container with ID starting with 7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5 not found: ID does not exist" containerID="7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5"
Jul 24 23:31:16.239583 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.239524    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5"} err="failed to get container status \"7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5\": rpc error: code = NotFound desc = could not find container \"7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5\": container with ID starting with 7e85c68f9dc4679b5f1fc4eb750237b78d116ca8316af4e146c408215c47cde5 not found: ID does not exist"
Jul 24 23:31:16.239583 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.239543    2568 scope.go:117] "RemoveContainer" containerID="9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d"
Jul 24 23:31:16.247264 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.247230    2568 scope.go:117] "RemoveContainer" containerID="9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d"
Jul 24 23:31:16.247530 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:31:16.247510    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d\": container with ID starting with 9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d not found: ID does not exist" containerID="9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d"
Jul 24 23:31:16.247587 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.247542    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d"} err="failed to get container status \"9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d\": rpc error: code = NotFound desc = could not find container \"9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d\": container with ID starting with 9b24c8229e722521a68ed12e6a5578783da24f8126ea4cdcea3113bc01b97d9d not found: ID does not exist"
Jul 24 23:31:16.251742 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.251717    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"]
Jul 24 23:31:16.253359 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.253337    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-183bb6-5d97cdd485-bvft2"]
Jul 24 23:31:16.262273 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.262239    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"]
Jul 24 23:31:16.266173 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:16.266151    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-183bb6-68ddc9f8fb-5trnw"]
Jul 24 23:31:17.259380 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:17.259348    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89f4c1e6-9685-4bed-b241-caefe46f9980" path="/var/lib/kubelet/pods/89f4c1e6-9685-4bed-b241-caefe46f9980/volumes"
Jul 24 23:31:17.259907 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:17.259701    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee03bc71-8ebe-460d-84ec-1759d1940159" path="/var/lib/kubelet/pods/ee03bc71-8ebe-460d-84ec-1759d1940159/volumes"
Jul 24 23:31:30.570825 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.570793    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"]
Jul 24 23:31:30.571232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571137    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89f4c1e6-9685-4bed-b241-caefe46f9980" containerName="maas-api"
Jul 24 23:31:30.571232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571149    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="89f4c1e6-9685-4bed-b241-caefe46f9980" containerName="maas-api"
Jul 24 23:31:30.571232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571172    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="382b962e-b6db-4030-882a-c13caf0fac63" containerName="payload-processing"
Jul 24 23:31:30.571232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571178    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="382b962e-b6db-4030-882a-c13caf0fac63" containerName="payload-processing"
Jul 24 23:31:30.571232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571184    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee03bc71-8ebe-460d-84ec-1759d1940159" containerName="payload-pre-processing"
Jul 24 23:31:30.571232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571190    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee03bc71-8ebe-460d-84ec-1759d1940159" containerName="payload-pre-processing"
Jul 24 23:31:30.571232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571199    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bf91a4c-b72c-4476-a33a-7096e62d6c25" containerName="payload-pre-processing"
Jul 24 23:31:30.571232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571204    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bf91a4c-b72c-4476-a33a-7096e62d6c25" containerName="payload-pre-processing"
Jul 24 23:31:30.571232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571212    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0968b08e-b107-49d7-bb8a-5bc99eeba885" containerName="maas-api"
Jul 24 23:31:30.571232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571217    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0968b08e-b107-49d7-bb8a-5bc99eeba885" containerName="maas-api"
Jul 24 23:31:30.571605 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571290    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0968b08e-b107-49d7-bb8a-5bc99eeba885" containerName="maas-api"
Jul 24 23:31:30.571605 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571300    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bf91a4c-b72c-4476-a33a-7096e62d6c25" containerName="payload-pre-processing"
Jul 24 23:31:30.571605 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571308    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="382b962e-b6db-4030-882a-c13caf0fac63" containerName="payload-processing"
Jul 24 23:31:30.571605 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571315    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="89f4c1e6-9685-4bed-b241-caefe46f9980" containerName="maas-api"
Jul 24 23:31:30.571605 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.571321    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee03bc71-8ebe-460d-84ec-1759d1940159" containerName="payload-pre-processing"
Jul 24 23:31:30.574152 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.574137    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:30.577784 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.577763    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-fe475f-serving-cert\""
Jul 24 23:31:30.596447 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.596424    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"]
Jul 24 23:31:30.623482 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.623457    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:30.623639 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.623488    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gt7c\" (UniqueName: \"kubernetes.io/projected/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-kube-api-access-4gt7c\") pod \"maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:30.623639 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.623524    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-maas-api-tls\") pod \"maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:30.724538 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.724515    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:30.724684 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.724542    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gt7c\" (UniqueName: \"kubernetes.io/projected/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-kube-api-access-4gt7c\") pod \"maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:30.724684 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.724579    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-maas-api-tls\") pod \"maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:30.725163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.725141    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:30.726846 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.726826    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-maas-api-tls\") pod \"maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:30.735420 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.735401    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gt7c\" (UniqueName: \"kubernetes.io/projected/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-kube-api-access-4gt7c\") pod \"maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:30.883772 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:30.883700    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:31.010889 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.010863    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"]
Jul 24 23:31:31.012449 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:31:31.012418    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cfb6d7f_53ea_458a_b0f9_dbdd8562e98f.slice/crio-04116c52ddb5314fafe064f5c7697afc6bbab4458132c5ef4fc995cf5e898a39 WatchSource:0}: Error finding container 04116c52ddb5314fafe064f5c7697afc6bbab4458132c5ef4fc995cf5e898a39: Status 404 returned error can't find the container with id 04116c52ddb5314fafe064f5c7697afc6bbab4458132c5ef4fc995cf5e898a39
Jul 24 23:31:31.275841 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.275816    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"]
Jul 24 23:31:31.280434 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.280411    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"]
Jul 24 23:31:31.280558 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.280517    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:31:31.283140 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.283122    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-fe475f-dockercfg-dbkgx\""
Jul 24 23:31:31.283140 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.283135    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-fe475f\""
Jul 24 23:31:31.283750 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.283735    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:31:31.290804 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.290775    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"]
Jul 24 23:31:31.297117 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.297087    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"]
Jul 24 23:31:31.303572 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.303546    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2" event={"ID":"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f","Type":"ContainerStarted","Data":"04116c52ddb5314fafe064f5c7697afc6bbab4458132c5ef4fc995cf5e898a39"}
Jul 24 23:31:31.330599 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.330577    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48a2fda1-34a1-4885-b67e-f1876ee6f404-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989\" (UID: \"48a2fda1-34a1-4885-b67e-f1876ee6f404\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:31:31.330711 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.330634    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6h74\" (UniqueName: \"kubernetes.io/projected/85be21c6-3865-4916-8c99-dc37fbacdf09-kube-api-access-c6h74\") pod \"payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp\" (UID: \"85be21c6-3865-4916-8c99-dc37fbacdf09\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:31:31.330711 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.330667    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ghr6\" (UniqueName: \"kubernetes.io/projected/48a2fda1-34a1-4885-b67e-f1876ee6f404-kube-api-access-4ghr6\") pod \"payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989\" (UID: \"48a2fda1-34a1-4885-b67e-f1876ee6f404\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:31:31.330788 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.330743    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85be21c6-3865-4916-8c99-dc37fbacdf09-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp\" (UID: \"85be21c6-3865-4916-8c99-dc37fbacdf09\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:31:31.431656 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.431609    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48a2fda1-34a1-4885-b67e-f1876ee6f404-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989\" (UID: \"48a2fda1-34a1-4885-b67e-f1876ee6f404\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:31:31.431746 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.431688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6h74\" (UniqueName: \"kubernetes.io/projected/85be21c6-3865-4916-8c99-dc37fbacdf09-kube-api-access-c6h74\") pod \"payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp\" (UID: \"85be21c6-3865-4916-8c99-dc37fbacdf09\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:31:31.431746 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.431714    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ghr6\" (UniqueName: \"kubernetes.io/projected/48a2fda1-34a1-4885-b67e-f1876ee6f404-kube-api-access-4ghr6\") pod \"payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989\" (UID: \"48a2fda1-34a1-4885-b67e-f1876ee6f404\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:31:31.431855 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.431759    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85be21c6-3865-4916-8c99-dc37fbacdf09-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp\" (UID: \"85be21c6-3865-4916-8c99-dc37fbacdf09\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:31:31.432457 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.432436    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48a2fda1-34a1-4885-b67e-f1876ee6f404-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989\" (UID: \"48a2fda1-34a1-4885-b67e-f1876ee6f404\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:31:31.432549 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.432479    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85be21c6-3865-4916-8c99-dc37fbacdf09-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp\" (UID: \"85be21c6-3865-4916-8c99-dc37fbacdf09\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:31:31.441676 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.441649    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6h74\" (UniqueName: \"kubernetes.io/projected/85be21c6-3865-4916-8c99-dc37fbacdf09-kube-api-access-c6h74\") pod \"payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp\" (UID: \"85be21c6-3865-4916-8c99-dc37fbacdf09\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:31:31.441976 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.441959    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ghr6\" (UniqueName: \"kubernetes.io/projected/48a2fda1-34a1-4885-b67e-f1876ee6f404-kube-api-access-4ghr6\") pod \"payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989\" (UID: \"48a2fda1-34a1-4885-b67e-f1876ee6f404\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:31:31.601299 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.601192    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:31:31.609512 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.609223    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:31:31.785780 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.785739    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"]
Jul 24 23:31:31.787927 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:31:31.787903    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85be21c6_3865_4916_8c99_dc37fbacdf09.slice/crio-d851e534ebc5421d4b69464a6630e49e97cc6f71bd31226312d7396a1cebf6b5 WatchSource:0}: Error finding container d851e534ebc5421d4b69464a6630e49e97cc6f71bd31226312d7396a1cebf6b5: Status 404 returned error can't find the container with id d851e534ebc5421d4b69464a6630e49e97cc6f71bd31226312d7396a1cebf6b5
Jul 24 23:31:31.798937 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:31.798912    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"]
Jul 24 23:31:31.801962 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:31:31.801933    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a2fda1_34a1_4885_b67e_f1876ee6f404.slice/crio-8bc30736fbadced468b00b38dd094c7438ac396c5e97ee36bfa106289fb42233 WatchSource:0}: Error finding container 8bc30736fbadced468b00b38dd094c7438ac396c5e97ee36bfa106289fb42233: Status 404 returned error can't find the container with id 8bc30736fbadced468b00b38dd094c7438ac396c5e97ee36bfa106289fb42233
Jul 24 23:31:32.309614 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.309579    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2" event={"ID":"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f","Type":"ContainerStarted","Data":"de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d"}
Jul 24 23:31:32.309831 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.309683    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:32.311128 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.311109    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989" event={"ID":"48a2fda1-34a1-4885-b67e-f1876ee6f404","Type":"ContainerStarted","Data":"cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb"}
Jul 24 23:31:32.311239 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.311151    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989" event={"ID":"48a2fda1-34a1-4885-b67e-f1876ee6f404","Type":"ContainerStarted","Data":"8bc30736fbadced468b00b38dd094c7438ac396c5e97ee36bfa106289fb42233"}
Jul 24 23:31:32.311239 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.311171    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:31:32.312504 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.312485    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp" event={"ID":"85be21c6-3865-4916-8c99-dc37fbacdf09","Type":"ContainerStarted","Data":"a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c"}
Jul 24 23:31:32.312585 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.312519    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp" event={"ID":"85be21c6-3865-4916-8c99-dc37fbacdf09","Type":"ContainerStarted","Data":"d851e534ebc5421d4b69464a6630e49e97cc6f71bd31226312d7396a1cebf6b5"}
Jul 24 23:31:32.312585 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.312540    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:31:32.328958 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.328907    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2" podStartSLOduration=1.910871661 podStartE2EDuration="2.328894157s" totalImagesPullingTime="418.022496ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:31:32.326429342 +0000 UTC m=+2349.603462885" watchObservedRunningTime="2026-07-24 23:31:32.328894157 +0000 UTC m=+2349.605927701"
Jul 24 23:31:32.344470 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.344421    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989" podStartSLOduration=1.344407505 podStartE2EDuration="1.344407505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:31:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:31:32.342821974 +0000 UTC m=+2349.619855547" watchObservedRunningTime="2026-07-24 23:31:32.344407505 +0000 UTC m=+2349.621441076"
Jul 24 23:31:32.359950 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:32.359912    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp" podStartSLOduration=1.359897277 podStartE2EDuration="1.359897277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:31:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:31:32.358478413 +0000 UTC m=+2349.635511982" watchObservedRunningTime="2026-07-24 23:31:32.359897277 +0000 UTC m=+2349.636930822"
Jul 24 23:31:38.322400 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.322371    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:31:38.362032 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.361997    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"]
Jul 24 23:31:38.365776 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.365753    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:38.368450 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.368428    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-6f3ce2-serving-cert\""
Jul 24 23:31:38.378558 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.378532    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"]
Jul 24 23:31:38.501171 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.501137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a19449e-401c-4edb-822b-795e502e6726-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:38.501348 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.501182    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4c2j\" (UniqueName: \"kubernetes.io/projected/5a19449e-401c-4edb-822b-795e502e6726-kube-api-access-g4c2j\") pod \"maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:38.501348 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.501212    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a19449e-401c-4edb-822b-795e502e6726-maas-api-tls\") pod \"maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:38.602266 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.602172    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a19449e-401c-4edb-822b-795e502e6726-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:38.602266 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.602223    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4c2j\" (UniqueName: \"kubernetes.io/projected/5a19449e-401c-4edb-822b-795e502e6726-kube-api-access-g4c2j\") pod \"maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:38.602473 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.602281    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a19449e-401c-4edb-822b-795e502e6726-maas-api-tls\") pod \"maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:38.602473 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:31:38.602423    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-serving-cert: secret "maas-api-e2e-shared-b-6f3ce2-serving-cert" not found
Jul 24 23:31:38.602574 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:31:38.602485    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a19449e-401c-4edb-822b-795e502e6726-maas-api-tls podName:5a19449e-401c-4edb-822b-795e502e6726 nodeName:}" failed. No retries permitted until 2026-07-24 23:31:39.102464016 +0000 UTC m=+2356.379497549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a19449e-401c-4edb-822b-795e502e6726-maas-api-tls") pod "maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s" (UID: "5a19449e-401c-4edb-822b-795e502e6726") : secret "maas-api-e2e-shared-b-6f3ce2-serving-cert" not found
Jul 24 23:31:38.602843 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.602823    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a19449e-401c-4edb-822b-795e502e6726-openshift-service-ca\") pod \"maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:38.612985 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:38.612964    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4c2j\" (UniqueName: \"kubernetes.io/projected/5a19449e-401c-4edb-822b-795e502e6726-kube-api-access-g4c2j\") pod \"maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:39.097614 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.097582    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"]
Jul 24 23:31:39.101071 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.101056    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:31:39.103683 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.103654    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-6f3ce2\""
Jul 24 23:31:39.107189 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.107167    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a19449e-401c-4edb-822b-795e502e6726-maas-api-tls\") pod \"maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:39.109548 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.109524    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a19449e-401c-4edb-822b-795e502e6726-maas-api-tls\") pod \"maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:39.113006 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.112983    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"]
Jul 24 23:31:39.208470 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.208431    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gqpt\" (UniqueName: \"kubernetes.io/projected/b9451d15-f546-40ac-8b94-2633b7a7e1fd-kube-api-access-4gqpt\") pod \"payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf\" (UID: \"b9451d15-f546-40ac-8b94-2633b7a7e1fd\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:31:39.208644 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.208485    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9451d15-f546-40ac-8b94-2633b7a7e1fd-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf\" (UID: \"b9451d15-f546-40ac-8b94-2633b7a7e1fd\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:31:39.279575 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.279547    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:39.309698 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.309669    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gqpt\" (UniqueName: \"kubernetes.io/projected/b9451d15-f546-40ac-8b94-2633b7a7e1fd-kube-api-access-4gqpt\") pod \"payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf\" (UID: \"b9451d15-f546-40ac-8b94-2633b7a7e1fd\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:31:39.309850 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.309721    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9451d15-f546-40ac-8b94-2633b7a7e1fd-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf\" (UID: \"b9451d15-f546-40ac-8b94-2633b7a7e1fd\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:31:39.310370 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.310351    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9451d15-f546-40ac-8b94-2633b7a7e1fd-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf\" (UID: \"b9451d15-f546-40ac-8b94-2633b7a7e1fd\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:31:39.320628 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.320600    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gqpt\" (UniqueName: \"kubernetes.io/projected/b9451d15-f546-40ac-8b94-2633b7a7e1fd-kube-api-access-4gqpt\") pod \"payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf\" (UID: \"b9451d15-f546-40ac-8b94-2633b7a7e1fd\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:31:39.411488 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.411226    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:31:39.411488 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.411283    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"]
Jul 24 23:31:39.413417 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:31:39.413396    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a19449e_401c_4edb_822b_795e502e6726.slice/crio-c13c612e98b7bd9f2d72af194bc657fe4ad3e57e13d8738bf68c66f4538b8231 WatchSource:0}: Error finding container c13c612e98b7bd9f2d72af194bc657fe4ad3e57e13d8738bf68c66f4538b8231: Status 404 returned error can't find the container with id c13c612e98b7bd9f2d72af194bc657fe4ad3e57e13d8738bf68c66f4538b8231
Jul 24 23:31:39.538461 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:39.538435    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"]
Jul 24 23:31:39.540584 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:31:39.540561    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9451d15_f546_40ac_8b94_2633b7a7e1fd.slice/crio-e99f610194572bf25b3a82aec8d2f7ee2850abf9d0e01b101ce4bfb97fb0562a WatchSource:0}: Error finding container e99f610194572bf25b3a82aec8d2f7ee2850abf9d0e01b101ce4bfb97fb0562a: Status 404 returned error can't find the container with id e99f610194572bf25b3a82aec8d2f7ee2850abf9d0e01b101ce4bfb97fb0562a
Jul 24 23:31:40.357993 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:40.357950    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s" event={"ID":"5a19449e-401c-4edb-822b-795e502e6726","Type":"ContainerStarted","Data":"48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301"}
Jul 24 23:31:40.358198 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:40.358065    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:40.358198 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:40.358122    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s" event={"ID":"5a19449e-401c-4edb-822b-795e502e6726","Type":"ContainerStarted","Data":"c13c612e98b7bd9f2d72af194bc657fe4ad3e57e13d8738bf68c66f4538b8231"}
Jul 24 23:31:40.359661 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:40.359635    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf" event={"ID":"b9451d15-f546-40ac-8b94-2633b7a7e1fd","Type":"ContainerStarted","Data":"28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30"}
Jul 24 23:31:40.359800 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:40.359688    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf" event={"ID":"b9451d15-f546-40ac-8b94-2633b7a7e1fd","Type":"ContainerStarted","Data":"e99f610194572bf25b3a82aec8d2f7ee2850abf9d0e01b101ce4bfb97fb0562a"}
Jul 24 23:31:40.359800 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:40.359720    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:31:40.375375 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:40.375334    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s" podStartSLOduration=2.079942508 podStartE2EDuration="2.375318537s" totalImagesPullingTime="295.376029ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:31:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:31:40.373481612 +0000 UTC m=+2357.650515155" watchObservedRunningTime="2026-07-24 23:31:40.375318537 +0000 UTC m=+2357.652352116"
Jul 24 23:31:40.391648 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:40.391591    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf" podStartSLOduration=1.391574169 podStartE2EDuration="1.391574169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:31:40.39104279 +0000 UTC m=+2357.668076356" watchObservedRunningTime="2026-07-24 23:31:40.391574169 +0000 UTC m=+2357.668607711"
Jul 24 23:31:43.318958 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:43.318929    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:31:43.319342 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:43.319007    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:31:46.369838 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:46.369761    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:31:51.366449 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:31:51.366416    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:38:10.689351 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.689318    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"]
Jul 24 23:38:10.693452 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.693434    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:10.696045 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.696028    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-0aa0e6-serving-cert\""
Jul 24 23:38:10.701800 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.701778    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"]
Jul 24 23:38:10.749908 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.749887    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/871c48ab-e84b-46fc-8f94-9f52ed81b99e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:10.750029 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.749938    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871c48ab-e84b-46fc-8f94-9f52ed81b99e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:10.750029 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.749996    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kshpl\" (UniqueName: \"kubernetes.io/projected/871c48ab-e84b-46fc-8f94-9f52ed81b99e-kube-api-access-kshpl\") pod \"maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:10.851205 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.851175    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871c48ab-e84b-46fc-8f94-9f52ed81b99e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:10.851370 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.851215    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kshpl\" (UniqueName: \"kubernetes.io/projected/871c48ab-e84b-46fc-8f94-9f52ed81b99e-kube-api-access-kshpl\") pod \"maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:10.851370 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.851283    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/871c48ab-e84b-46fc-8f94-9f52ed81b99e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:10.851370 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:38:10.851359    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-serving-cert: secret "maas-api-e2e-ipp-a-0aa0e6-serving-cert" not found
Jul 24 23:38:10.851534 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:38:10.851447    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/871c48ab-e84b-46fc-8f94-9f52ed81b99e-maas-api-tls podName:871c48ab-e84b-46fc-8f94-9f52ed81b99e nodeName:}" failed. No retries permitted until 2026-07-24 23:38:11.3514246 +0000 UTC m=+2748.628458125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/871c48ab-e84b-46fc-8f94-9f52ed81b99e-maas-api-tls") pod "maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm" (UID: "871c48ab-e84b-46fc-8f94-9f52ed81b99e") : secret "maas-api-e2e-ipp-a-0aa0e6-serving-cert" not found
Jul 24 23:38:10.856207 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.852364    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/871c48ab-e84b-46fc-8f94-9f52ed81b99e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:10.862009 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:10.861986    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kshpl\" (UniqueName: \"kubernetes.io/projected/871c48ab-e84b-46fc-8f94-9f52ed81b99e-kube-api-access-kshpl\") pod \"maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:11.355038 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.355004    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871c48ab-e84b-46fc-8f94-9f52ed81b99e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:11.357490 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.357466    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871c48ab-e84b-46fc-8f94-9f52ed81b99e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:11.554624 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.554595    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"]
Jul 24 23:38:11.558352 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.558328    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:38:11.561909 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.561886    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-0aa0e6-dockercfg-vt6hp\""
Jul 24 23:38:11.562340 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.562324    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-0aa0e6\""
Jul 24 23:38:11.570613 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.570589    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"]
Jul 24 23:38:11.606064 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.606003    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:11.658648 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.658564    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbmmq\" (UniqueName: \"kubernetes.io/projected/b68d9682-be9f-4612-8603-bc549c38e85e-kube-api-access-lbmmq\") pod \"payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc\" (UID: \"b68d9682-be9f-4612-8603-bc549c38e85e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:38:11.658806 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.658742    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b68d9682-be9f-4612-8603-bc549c38e85e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc\" (UID: \"b68d9682-be9f-4612-8603-bc549c38e85e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:38:11.742806 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.742780    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"]
Jul 24 23:38:11.743792 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:38:11.743761    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod871c48ab_e84b_46fc_8f94_9f52ed81b99e.slice/crio-03ab23a376b122418d402b5bbb3b06ec8387276b3cf996ab57497aafc352ad62 WatchSource:0}: Error finding container 03ab23a376b122418d402b5bbb3b06ec8387276b3cf996ab57497aafc352ad62: Status 404 returned error can't find the container with id 03ab23a376b122418d402b5bbb3b06ec8387276b3cf996ab57497aafc352ad62
Jul 24 23:38:11.745542 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.745525    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 23:38:11.759598 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.759577    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbmmq\" (UniqueName: \"kubernetes.io/projected/b68d9682-be9f-4612-8603-bc549c38e85e-kube-api-access-lbmmq\") pod \"payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc\" (UID: \"b68d9682-be9f-4612-8603-bc549c38e85e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:38:11.759734 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.759701    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b68d9682-be9f-4612-8603-bc549c38e85e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc\" (UID: \"b68d9682-be9f-4612-8603-bc549c38e85e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:38:11.760607 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.760590    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b68d9682-be9f-4612-8603-bc549c38e85e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc\" (UID: \"b68d9682-be9f-4612-8603-bc549c38e85e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:38:11.771569 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.771551    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbmmq\" (UniqueName: \"kubernetes.io/projected/b68d9682-be9f-4612-8603-bc549c38e85e-kube-api-access-lbmmq\") pod \"payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc\" (UID: \"b68d9682-be9f-4612-8603-bc549c38e85e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:38:11.869783 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:11.869733    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:38:12.016015 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:12.015985    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"]
Jul 24 23:38:12.017389 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:38:12.017363    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb68d9682_be9f_4612_8603_bc549c38e85e.slice/crio-5dd281ad9c68274b0aaeb00488765f33fe22c6766e83a77e04af21347cb9ec1a WatchSource:0}: Error finding container 5dd281ad9c68274b0aaeb00488765f33fe22c6766e83a77e04af21347cb9ec1a: Status 404 returned error can't find the container with id 5dd281ad9c68274b0aaeb00488765f33fe22c6766e83a77e04af21347cb9ec1a
Jul 24 23:38:12.236518 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:12.236491    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm" event={"ID":"871c48ab-e84b-46fc-8f94-9f52ed81b99e","Type":"ContainerStarted","Data":"03ab23a376b122418d402b5bbb3b06ec8387276b3cf996ab57497aafc352ad62"}
Jul 24 23:38:12.237853 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:12.237831    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc" event={"ID":"b68d9682-be9f-4612-8603-bc549c38e85e","Type":"ContainerStarted","Data":"564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb"}
Jul 24 23:38:12.237947 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:12.237870    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc" event={"ID":"b68d9682-be9f-4612-8603-bc549c38e85e","Type":"ContainerStarted","Data":"5dd281ad9c68274b0aaeb00488765f33fe22c6766e83a77e04af21347cb9ec1a"}
Jul 24 23:38:12.237989 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:12.237962    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:38:12.269955 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:12.269909    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc" podStartSLOduration=1.269889424 podStartE2EDuration="1.269889424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:38:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:38:12.267988272 +0000 UTC m=+2749.545021843" watchObservedRunningTime="2026-07-24 23:38:12.269889424 +0000 UTC m=+2749.546922969"
Jul 24 23:38:13.244358 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:13.244321    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm" event={"ID":"871c48ab-e84b-46fc-8f94-9f52ed81b99e","Type":"ContainerStarted","Data":"ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8"}
Jul 24 23:38:13.244795 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:13.244406    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:13.262926 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:13.262850    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm" podStartSLOduration=2.8361281739999997 podStartE2EDuration="3.262831489s" totalImagesPullingTime="426.703315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:38:13.262325492 +0000 UTC m=+2750.539359035" watchObservedRunningTime="2026-07-24 23:38:13.262831489 +0000 UTC m=+2750.539865034"
Jul 24 23:38:19.258575 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:19.258549    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:38:23.246441 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:23.246408    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:38:30.074825 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.074794    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"]
Jul 24 23:38:30.078769 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.078755    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:38:30.081416 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.081390    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-3fb847\""
Jul 24 23:38:30.081416 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.081406    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-3fb847-dockercfg-gvd8l\""
Jul 24 23:38:30.088290 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.088268    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"]
Jul 24 23:38:30.222608 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.222580    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg9br\" (UniqueName: \"kubernetes.io/projected/1e84c4ca-e30b-4f94-8b67-9b6479144183-kube-api-access-zg9br\") pod \"payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr\" (UID: \"1e84c4ca-e30b-4f94-8b67-9b6479144183\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:38:30.222777 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.222693    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e84c4ca-e30b-4f94-8b67-9b6479144183-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr\" (UID: \"1e84c4ca-e30b-4f94-8b67-9b6479144183\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:38:30.323635 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.323601    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg9br\" (UniqueName: \"kubernetes.io/projected/1e84c4ca-e30b-4f94-8b67-9b6479144183-kube-api-access-zg9br\") pod \"payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr\" (UID: \"1e84c4ca-e30b-4f94-8b67-9b6479144183\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:38:30.323819 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.323694    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e84c4ca-e30b-4f94-8b67-9b6479144183-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr\" (UID: \"1e84c4ca-e30b-4f94-8b67-9b6479144183\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:38:30.324403 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.324385    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e84c4ca-e30b-4f94-8b67-9b6479144183-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr\" (UID: \"1e84c4ca-e30b-4f94-8b67-9b6479144183\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:38:30.331984 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.331922    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg9br\" (UniqueName: \"kubernetes.io/projected/1e84c4ca-e30b-4f94-8b67-9b6479144183-kube-api-access-zg9br\") pod \"payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr\" (UID: \"1e84c4ca-e30b-4f94-8b67-9b6479144183\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:38:30.389566 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.389539    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:38:30.526820 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:30.526792    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"]
Jul 24 23:38:30.527732 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:38:30.527693    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e84c4ca_e30b_4f94_8b67_9b6479144183.slice/crio-a2677349e7f989dab2980cce110799acbd1c0168f07a572e8fa54444831f8c3d WatchSource:0}: Error finding container a2677349e7f989dab2980cce110799acbd1c0168f07a572e8fa54444831f8c3d: Status 404 returned error can't find the container with id a2677349e7f989dab2980cce110799acbd1c0168f07a572e8fa54444831f8c3d
Jul 24 23:38:31.337021 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:31.336991    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr" event={"ID":"1e84c4ca-e30b-4f94-8b67-9b6479144183","Type":"ContainerStarted","Data":"1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07"}
Jul 24 23:38:31.337497 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:31.337046    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr" event={"ID":"1e84c4ca-e30b-4f94-8b67-9b6479144183","Type":"ContainerStarted","Data":"a2677349e7f989dab2980cce110799acbd1c0168f07a572e8fa54444831f8c3d"}
Jul 24 23:38:31.337497 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:31.337115    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:38:31.356673 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:31.356626    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr" podStartSLOduration=1.356612352 podStartE2EDuration="1.356612352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:38:31.353772415 +0000 UTC m=+2768.630805958" watchObservedRunningTime="2026-07-24 23:38:31.356612352 +0000 UTC m=+2768.633645896"
Jul 24 23:38:42.344075 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:42.344046    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:38:50.957679 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:38:50.957599    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr_1e84c4ca-e30b-4f94-8b67-9b6479144183/payload-processing/0.log"
Jul 24 23:39:37.749765 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:37.749733    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc_b68d9682-be9f-4612-8603-bc549c38e85e/payload-processing/0.log"
Jul 24 23:39:37.993426 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:37.993397    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr_1e84c4ca-e30b-4f94-8b67-9b6479144183/payload-processing/0.log"
Jul 24 23:39:41.550606 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.550576    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"]
Jul 24 23:39:41.554517 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.554492    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:41.557084 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.557064    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-4bb47e-serving-cert\""
Jul 24 23:39:41.565273 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.565240    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"]
Jul 24 23:39:41.680180 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.680148    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5100dd3-83a6-4ee0-8402-e35a09f5d208-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:41.680180 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.680182    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5100dd3-83a6-4ee0-8402-e35a09f5d208-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:41.680417 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.680274    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzpm9\" (UniqueName: \"kubernetes.io/projected/a5100dd3-83a6-4ee0-8402-e35a09f5d208-kube-api-access-gzpm9\") pod \"maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:41.781544 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.781517    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzpm9\" (UniqueName: \"kubernetes.io/projected/a5100dd3-83a6-4ee0-8402-e35a09f5d208-kube-api-access-gzpm9\") pod \"maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:41.781735 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.781615    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5100dd3-83a6-4ee0-8402-e35a09f5d208-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:41.781735 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.781637    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5100dd3-83a6-4ee0-8402-e35a09f5d208-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:41.781849 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:39:41.781748    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-serving-cert: secret "maas-api-e2e-ipp-cleanup-4bb47e-serving-cert" not found
Jul 24 23:39:41.781849 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:39:41.781821    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5100dd3-83a6-4ee0-8402-e35a09f5d208-maas-api-tls podName:a5100dd3-83a6-4ee0-8402-e35a09f5d208 nodeName:}" failed. No retries permitted until 2026-07-24 23:39:42.281802177 +0000 UTC m=+2839.558835698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a5100dd3-83a6-4ee0-8402-e35a09f5d208-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz" (UID: "a5100dd3-83a6-4ee0-8402-e35a09f5d208") : secret "maas-api-e2e-ipp-cleanup-4bb47e-serving-cert" not found
Jul 24 23:39:41.782332 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.782312    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5100dd3-83a6-4ee0-8402-e35a09f5d208-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:41.793207 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:41.793189    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzpm9\" (UniqueName: \"kubernetes.io/projected/a5100dd3-83a6-4ee0-8402-e35a09f5d208-kube-api-access-gzpm9\") pod \"maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:42.271750 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.271698    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"]
Jul 24 23:39:42.276179 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.276160    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:39:42.279259 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.279230    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-4bb47e-dockercfg-fg5lw\""
Jul 24 23:39:42.279372 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.279230    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-4bb47e\""
Jul 24 23:39:42.286426 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.286401    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5100dd3-83a6-4ee0-8402-e35a09f5d208-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:42.289009 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.288987    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5100dd3-83a6-4ee0-8402-e35a09f5d208-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:42.291241 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.291221    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"]
Jul 24 23:39:42.387703 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.387663    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b95b46d-8d01-4e7c-a187-44322a6a7453-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8\" (UID: \"5b95b46d-8d01-4e7c-a187-44322a6a7453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:39:42.387882 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.387789    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcx5x\" (UniqueName: \"kubernetes.io/projected/5b95b46d-8d01-4e7c-a187-44322a6a7453-kube-api-access-kcx5x\") pod \"payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8\" (UID: \"5b95b46d-8d01-4e7c-a187-44322a6a7453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:39:42.469213 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.469181    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:42.488509 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.488480    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b95b46d-8d01-4e7c-a187-44322a6a7453-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8\" (UID: \"5b95b46d-8d01-4e7c-a187-44322a6a7453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:39:42.488626 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.488590    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcx5x\" (UniqueName: \"kubernetes.io/projected/5b95b46d-8d01-4e7c-a187-44322a6a7453-kube-api-access-kcx5x\") pod \"payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8\" (UID: \"5b95b46d-8d01-4e7c-a187-44322a6a7453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:39:42.489218 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.489197    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b95b46d-8d01-4e7c-a187-44322a6a7453-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8\" (UID: \"5b95b46d-8d01-4e7c-a187-44322a6a7453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:39:42.496975 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.496955    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcx5x\" (UniqueName: \"kubernetes.io/projected/5b95b46d-8d01-4e7c-a187-44322a6a7453-kube-api-access-kcx5x\") pod \"payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8\" (UID: \"5b95b46d-8d01-4e7c-a187-44322a6a7453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:39:42.586611 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.586581    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:39:42.603443 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.603420    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"]
Jul 24 23:39:42.605507 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:39:42.605481    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5100dd3_83a6_4ee0_8402_e35a09f5d208.slice/crio-4eefd903de37a934e036c191a7b1f614725deefea3ec0ea2f52692706d0def29 WatchSource:0}: Error finding container 4eefd903de37a934e036c191a7b1f614725deefea3ec0ea2f52692706d0def29: Status 404 returned error can't find the container with id 4eefd903de37a934e036c191a7b1f614725deefea3ec0ea2f52692706d0def29
Jul 24 23:39:42.694825 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.694690    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz" event={"ID":"a5100dd3-83a6-4ee0-8402-e35a09f5d208","Type":"ContainerStarted","Data":"4eefd903de37a934e036c191a7b1f614725deefea3ec0ea2f52692706d0def29"}
Jul 24 23:39:42.730772 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:42.730749    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"]
Jul 24 23:39:42.732483 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:39:42.732459    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b95b46d_8d01_4e7c_a187_44322a6a7453.slice/crio-41fd56083ad5f70aa55dc3adcf6c142267234db11110f5e6f4bd0449ea6c9c2c WatchSource:0}: Error finding container 41fd56083ad5f70aa55dc3adcf6c142267234db11110f5e6f4bd0449ea6c9c2c: Status 404 returned error can't find the container with id 41fd56083ad5f70aa55dc3adcf6c142267234db11110f5e6f4bd0449ea6c9c2c
Jul 24 23:39:43.700675 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:43.700643    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8" event={"ID":"5b95b46d-8d01-4e7c-a187-44322a6a7453","Type":"ContainerStarted","Data":"bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c"}
Jul 24 23:39:43.701138 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:43.700691    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8" event={"ID":"5b95b46d-8d01-4e7c-a187-44322a6a7453","Type":"ContainerStarted","Data":"41fd56083ad5f70aa55dc3adcf6c142267234db11110f5e6f4bd0449ea6c9c2c"}
Jul 24 23:39:43.701138 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:43.700751    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:39:43.702200 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:43.702180    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz" event={"ID":"a5100dd3-83a6-4ee0-8402-e35a09f5d208","Type":"ContainerStarted","Data":"62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff"}
Jul 24 23:39:43.702311 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:43.702232    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:43.717684 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:43.717644    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8" podStartSLOduration=1.717632421 podStartE2EDuration="1.717632421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:39:43.717482085 +0000 UTC m=+2840.994515647" watchObservedRunningTime="2026-07-24 23:39:43.717632421 +0000 UTC m=+2840.994665964"
Jul 24 23:39:43.737511 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:43.737458    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz" podStartSLOduration=2.368782215 podStartE2EDuration="2.737440925s" totalImagesPullingTime="368.65871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:39:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:39:43.734130924 +0000 UTC m=+2841.011164467" watchObservedRunningTime="2026-07-24 23:39:43.737440925 +0000 UTC m=+2841.014474469"
Jul 24 23:39:49.712300 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:49.712238    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:39:54.708697 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:39:54.708664    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:40:07.408393 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.408354    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"]
Jul 24 23:40:07.408859 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.408599    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz" podUID="a5100dd3-83a6-4ee0-8402-e35a09f5d208" containerName="maas-api" containerID="cri-o://62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff" gracePeriod=30
Jul 24 23:40:07.461405 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.461378    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"]
Jul 24 23:40:07.461603 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.461582    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8" podUID="5b95b46d-8d01-4e7c-a187-44322a6a7453" containerName="payload-processing" containerID="cri-o://bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c" gracePeriod=30
Jul 24 23:40:07.703053 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.703027    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:40:07.726304 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.726280    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzpm9\" (UniqueName: \"kubernetes.io/projected/a5100dd3-83a6-4ee0-8402-e35a09f5d208-kube-api-access-gzpm9\") pod \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") "
Jul 24 23:40:07.726440 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.726340    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5100dd3-83a6-4ee0-8402-e35a09f5d208-maas-api-tls\") pod \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") "
Jul 24 23:40:07.726440 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.726393    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5100dd3-83a6-4ee0-8402-e35a09f5d208-openshift-service-ca\") pod \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\" (UID: \"a5100dd3-83a6-4ee0-8402-e35a09f5d208\") "
Jul 24 23:40:07.727074 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.727001    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5100dd3-83a6-4ee0-8402-e35a09f5d208-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5100dd3-83a6-4ee0-8402-e35a09f5d208" (UID: "a5100dd3-83a6-4ee0-8402-e35a09f5d208"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:40:07.728734 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.728709    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5100dd3-83a6-4ee0-8402-e35a09f5d208-kube-api-access-gzpm9" (OuterVolumeSpecName: "kube-api-access-gzpm9") pod "a5100dd3-83a6-4ee0-8402-e35a09f5d208" (UID: "a5100dd3-83a6-4ee0-8402-e35a09f5d208"). InnerVolumeSpecName "kube-api-access-gzpm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:40:07.728839 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.728809    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5100dd3-83a6-4ee0-8402-e35a09f5d208-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5100dd3-83a6-4ee0-8402-e35a09f5d208" (UID: "a5100dd3-83a6-4ee0-8402-e35a09f5d208"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:40:07.735340 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.735311    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:40:07.827094 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827064    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b95b46d-8d01-4e7c-a187-44322a6a7453-plugins-config-volume\") pod \"5b95b46d-8d01-4e7c-a187-44322a6a7453\" (UID: \"5b95b46d-8d01-4e7c-a187-44322a6a7453\") "
Jul 24 23:40:07.827241 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827117    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcx5x\" (UniqueName: \"kubernetes.io/projected/5b95b46d-8d01-4e7c-a187-44322a6a7453-kube-api-access-kcx5x\") pod \"5b95b46d-8d01-4e7c-a187-44322a6a7453\" (UID: \"5b95b46d-8d01-4e7c-a187-44322a6a7453\") "
Jul 24 23:40:07.827438 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827414    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b95b46d-8d01-4e7c-a187-44322a6a7453-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b95b46d-8d01-4e7c-a187-44322a6a7453" (UID: "5b95b46d-8d01-4e7c-a187-44322a6a7453"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:40:07.827522 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827500    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzpm9\" (UniqueName: \"kubernetes.io/projected/a5100dd3-83a6-4ee0-8402-e35a09f5d208-kube-api-access-gzpm9\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:40:07.827577 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827527    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5100dd3-83a6-4ee0-8402-e35a09f5d208-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:40:07.827577 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827541    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b95b46d-8d01-4e7c-a187-44322a6a7453-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:40:07.827577 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827553    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5100dd3-83a6-4ee0-8402-e35a09f5d208-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:40:07.827737 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827699    2568 generic.go:356] "Generic (PLEG): container finished" podID="5b95b46d-8d01-4e7c-a187-44322a6a7453" containerID="bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c" exitCode=0
Jul 24 23:40:07.827775 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827745    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8" event={"ID":"5b95b46d-8d01-4e7c-a187-44322a6a7453","Type":"ContainerDied","Data":"bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c"}
Jul 24 23:40:07.827809 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827772    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"
Jul 24 23:40:07.827809 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827791    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8" event={"ID":"5b95b46d-8d01-4e7c-a187-44322a6a7453","Type":"ContainerDied","Data":"41fd56083ad5f70aa55dc3adcf6c142267234db11110f5e6f4bd0449ea6c9c2c"}
Jul 24 23:40:07.827907 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.827814    2568 scope.go:117] "RemoveContainer" containerID="bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c"
Jul 24 23:40:07.829319 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.829295    2568 generic.go:356] "Generic (PLEG): container finished" podID="a5100dd3-83a6-4ee0-8402-e35a09f5d208" containerID="62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff" exitCode=0
Jul 24 23:40:07.829435 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.829325    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz" event={"ID":"a5100dd3-83a6-4ee0-8402-e35a09f5d208","Type":"ContainerDied","Data":"62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff"}
Jul 24 23:40:07.829435 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.829372    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz" event={"ID":"a5100dd3-83a6-4ee0-8402-e35a09f5d208","Type":"ContainerDied","Data":"4eefd903de37a934e036c191a7b1f614725deefea3ec0ea2f52692706d0def29"}
Jul 24 23:40:07.829435 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.829335    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"
Jul 24 23:40:07.829605 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.829385    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b95b46d-8d01-4e7c-a187-44322a6a7453-kube-api-access-kcx5x" (OuterVolumeSpecName: "kube-api-access-kcx5x") pod "5b95b46d-8d01-4e7c-a187-44322a6a7453" (UID: "5b95b46d-8d01-4e7c-a187-44322a6a7453"). InnerVolumeSpecName "kube-api-access-kcx5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:40:07.836782 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.836764    2568 scope.go:117] "RemoveContainer" containerID="bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c"
Jul 24 23:40:07.837070 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:40:07.837052    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c\": container with ID starting with bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c not found: ID does not exist" containerID="bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c"
Jul 24 23:40:07.837154 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.837081    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c"} err="failed to get container status \"bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c\": rpc error: code = NotFound desc = could not find container \"bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c\": container with ID starting with bfe639565a6be3a1a9e035de2d2688c1c93c190c60de503bbe4060575684f35c not found: ID does not exist"
Jul 24 23:40:07.837154 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.837105    2568 scope.go:117] "RemoveContainer" containerID="62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff"
Jul 24 23:40:07.844564 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.844545    2568 scope.go:117] "RemoveContainer" containerID="62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff"
Jul 24 23:40:07.844781 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:40:07.844764    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff\": container with ID starting with 62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff not found: ID does not exist" containerID="62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff"
Jul 24 23:40:07.844844 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.844789    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff"} err="failed to get container status \"62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff\": rpc error: code = NotFound desc = could not find container \"62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff\": container with ID starting with 62cee65ffb6ebd775d34ab474122d59557dc300e9386d8a8fede2d791facf6ff not found: ID does not exist"
Jul 24 23:40:07.852442 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.852420    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"]
Jul 24 23:40:07.856219 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.856200    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4bb47e-7b96dffdf4-kb4xz"]
Jul 24 23:40:07.928769 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:07.928719    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcx5x\" (UniqueName: \"kubernetes.io/projected/5b95b46d-8d01-4e7c-a187-44322a6a7453-kube-api-access-kcx5x\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:40:08.150073 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:08.150035    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"]
Jul 24 23:40:08.152226 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:08.152200    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4bb47e-69f7458d6d-4gmn8"]
Jul 24 23:40:09.257867 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:09.257834    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b95b46d-8d01-4e7c-a187-44322a6a7453" path="/var/lib/kubelet/pods/5b95b46d-8d01-4e7c-a187-44322a6a7453/volumes"
Jul 24 23:40:09.258227 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:09.258196    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5100dd3-83a6-4ee0-8402-e35a09f5d208" path="/var/lib/kubelet/pods/a5100dd3-83a6-4ee0-8402-e35a09f5d208/volumes"
Jul 24 23:40:37.504666 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.504632    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"]
Jul 24 23:40:37.505140 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.504871    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm" podUID="871c48ab-e84b-46fc-8f94-9f52ed81b99e" containerName="maas-api" containerID="cri-o://ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8" gracePeriod=30
Jul 24 23:40:37.557969 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.557936    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"]
Jul 24 23:40:37.558229 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.558203    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc" podUID="b68d9682-be9f-4612-8603-bc549c38e85e" containerName="payload-processing" containerID="cri-o://564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb" gracePeriod=30
Jul 24 23:40:37.809689 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.809666    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:40:37.838129 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.838108    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:40:37.899775 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.899743    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871c48ab-e84b-46fc-8f94-9f52ed81b99e-maas-api-tls\") pod \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") "
Jul 24 23:40:37.899939 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.899783    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/871c48ab-e84b-46fc-8f94-9f52ed81b99e-openshift-service-ca\") pod \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") "
Jul 24 23:40:37.899939 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.899829    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kshpl\" (UniqueName: \"kubernetes.io/projected/871c48ab-e84b-46fc-8f94-9f52ed81b99e-kube-api-access-kshpl\") pod \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\" (UID: \"871c48ab-e84b-46fc-8f94-9f52ed81b99e\") "
Jul 24 23:40:37.900234 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.900207    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/871c48ab-e84b-46fc-8f94-9f52ed81b99e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "871c48ab-e84b-46fc-8f94-9f52ed81b99e" (UID: "871c48ab-e84b-46fc-8f94-9f52ed81b99e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:40:37.901956 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.901930    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/871c48ab-e84b-46fc-8f94-9f52ed81b99e-kube-api-access-kshpl" (OuterVolumeSpecName: "kube-api-access-kshpl") pod "871c48ab-e84b-46fc-8f94-9f52ed81b99e" (UID: "871c48ab-e84b-46fc-8f94-9f52ed81b99e"). InnerVolumeSpecName "kube-api-access-kshpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:40:37.902065 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.901968    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/871c48ab-e84b-46fc-8f94-9f52ed81b99e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "871c48ab-e84b-46fc-8f94-9f52ed81b99e" (UID: "871c48ab-e84b-46fc-8f94-9f52ed81b99e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:40:37.976283 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.976237    2568 generic.go:356] "Generic (PLEG): container finished" podID="b68d9682-be9f-4612-8603-bc549c38e85e" containerID="564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb" exitCode=0
Jul 24 23:40:37.976417 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.976297    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"
Jul 24 23:40:37.976417 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.976340    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc" event={"ID":"b68d9682-be9f-4612-8603-bc549c38e85e","Type":"ContainerDied","Data":"564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb"}
Jul 24 23:40:37.976417 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.976392    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc" event={"ID":"b68d9682-be9f-4612-8603-bc549c38e85e","Type":"ContainerDied","Data":"5dd281ad9c68274b0aaeb00488765f33fe22c6766e83a77e04af21347cb9ec1a"}
Jul 24 23:40:37.976585 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.976420    2568 scope.go:117] "RemoveContainer" containerID="564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb"
Jul 24 23:40:37.977652 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.977632    2568 generic.go:356] "Generic (PLEG): container finished" podID="871c48ab-e84b-46fc-8f94-9f52ed81b99e" containerID="ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8" exitCode=0
Jul 24 23:40:37.977714 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.977676    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"
Jul 24 23:40:37.977751 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.977710    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm" event={"ID":"871c48ab-e84b-46fc-8f94-9f52ed81b99e","Type":"ContainerDied","Data":"ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8"}
Jul 24 23:40:37.977751 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.977741    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm" event={"ID":"871c48ab-e84b-46fc-8f94-9f52ed81b99e","Type":"ContainerDied","Data":"03ab23a376b122418d402b5bbb3b06ec8387276b3cf996ab57497aafc352ad62"}
Jul 24 23:40:37.985052 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.985035    2568 scope.go:117] "RemoveContainer" containerID="564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb"
Jul 24 23:40:37.985367 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:40:37.985307    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb\": container with ID starting with 564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb not found: ID does not exist" containerID="564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb"
Jul 24 23:40:37.985367 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.985340    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb"} err="failed to get container status \"564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb\": rpc error: code = NotFound desc = could not find container \"564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb\": container with ID starting with 564a7c25de9f9e9e8fe66266fe003910c6249893ab5bacfe52fd084a4c4c24cb not found: ID does not exist"
Jul 24 23:40:37.985367 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.985362    2568 scope.go:117] "RemoveContainer" containerID="ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8"
Jul 24 23:40:37.994882 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.993669    2568 scope.go:117] "RemoveContainer" containerID="ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8"
Jul 24 23:40:37.995125 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:40:37.995108    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8\": container with ID starting with ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8 not found: ID does not exist" containerID="ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8"
Jul 24 23:40:37.995184 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:37.995135    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8"} err="failed to get container status \"ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8\": rpc error: code = NotFound desc = could not find container \"ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8\": container with ID starting with ea2dc98522de2ed6bff8ee057a97d7773c4af8a6990fe044e7f32eaab93733b8 not found: ID does not exist"
Jul 24 23:40:38.000804 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.000614    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"]
Jul 24 23:40:38.000804 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.000713    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b68d9682-be9f-4612-8603-bc549c38e85e-plugins-config-volume\") pod \"b68d9682-be9f-4612-8603-bc549c38e85e\" (UID: \"b68d9682-be9f-4612-8603-bc549c38e85e\") "
Jul 24 23:40:38.000943 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.000855    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbmmq\" (UniqueName: \"kubernetes.io/projected/b68d9682-be9f-4612-8603-bc549c38e85e-kube-api-access-lbmmq\") pod \"b68d9682-be9f-4612-8603-bc549c38e85e\" (UID: \"b68d9682-be9f-4612-8603-bc549c38e85e\") "
Jul 24 23:40:38.001136 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.001113    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b68d9682-be9f-4612-8603-bc549c38e85e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b68d9682-be9f-4612-8603-bc549c38e85e" (UID: "b68d9682-be9f-4612-8603-bc549c38e85e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:40:38.001210 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.001193    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871c48ab-e84b-46fc-8f94-9f52ed81b99e-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:40:38.001263 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.001214    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/871c48ab-e84b-46fc-8f94-9f52ed81b99e-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:40:38.001263 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.001224    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b68d9682-be9f-4612-8603-bc549c38e85e-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:40:38.001263 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.001232    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kshpl\" (UniqueName: \"kubernetes.io/projected/871c48ab-e84b-46fc-8f94-9f52ed81b99e-kube-api-access-kshpl\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:40:38.002732 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.002710    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b68d9682-be9f-4612-8603-bc549c38e85e-kube-api-access-lbmmq" (OuterVolumeSpecName: "kube-api-access-lbmmq") pod "b68d9682-be9f-4612-8603-bc549c38e85e" (UID: "b68d9682-be9f-4612-8603-bc549c38e85e"). InnerVolumeSpecName "kube-api-access-lbmmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:40:38.002785 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.002742    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0aa0e6-6b9d7d587d-b8wlm"]
Jul 24 23:40:38.102716 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.102643    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbmmq\" (UniqueName: \"kubernetes.io/projected/b68d9682-be9f-4612-8603-bc549c38e85e-kube-api-access-lbmmq\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:40:38.297724 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.297697    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"]
Jul 24 23:40:38.299430 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:38.299409    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0aa0e6-6dcb97576-qjqnc"]
Jul 24 23:40:39.257846 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:39.257815    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="871c48ab-e84b-46fc-8f94-9f52ed81b99e" path="/var/lib/kubelet/pods/871c48ab-e84b-46fc-8f94-9f52ed81b99e/volumes"
Jul 24 23:40:39.258217 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:40:39.258178    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b68d9682-be9f-4612-8603-bc549c38e85e" path="/var/lib/kubelet/pods/b68d9682-be9f-4612-8603-bc549c38e85e/volumes"
Jul 24 23:41:46.658310 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:46.658218    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"]
Jul 24 23:41:46.658861 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:46.658511    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr" podUID="1e84c4ca-e30b-4f94-8b67-9b6479144183" containerName="payload-processing" containerID="cri-o://1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07" gracePeriod=30
Jul 24 23:41:46.921756 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:46.921700    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:41:47.018709 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.018680    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg9br\" (UniqueName: \"kubernetes.io/projected/1e84c4ca-e30b-4f94-8b67-9b6479144183-kube-api-access-zg9br\") pod \"1e84c4ca-e30b-4f94-8b67-9b6479144183\" (UID: \"1e84c4ca-e30b-4f94-8b67-9b6479144183\") "
Jul 24 23:41:47.018883 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.018807    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e84c4ca-e30b-4f94-8b67-9b6479144183-plugins-config-volume\") pod \"1e84c4ca-e30b-4f94-8b67-9b6479144183\" (UID: \"1e84c4ca-e30b-4f94-8b67-9b6479144183\") "
Jul 24 23:41:47.019215 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.019184    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e84c4ca-e30b-4f94-8b67-9b6479144183-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1e84c4ca-e30b-4f94-8b67-9b6479144183" (UID: "1e84c4ca-e30b-4f94-8b67-9b6479144183"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:41:47.020841 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.020816    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e84c4ca-e30b-4f94-8b67-9b6479144183-kube-api-access-zg9br" (OuterVolumeSpecName: "kube-api-access-zg9br") pod "1e84c4ca-e30b-4f94-8b67-9b6479144183" (UID: "1e84c4ca-e30b-4f94-8b67-9b6479144183"). InnerVolumeSpecName "kube-api-access-zg9br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:41:47.120608 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.120575    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e84c4ca-e30b-4f94-8b67-9b6479144183-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:41:47.120608 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.120602    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg9br\" (UniqueName: \"kubernetes.io/projected/1e84c4ca-e30b-4f94-8b67-9b6479144183-kube-api-access-zg9br\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:41:47.306163 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.306134    2568 generic.go:356] "Generic (PLEG): container finished" podID="1e84c4ca-e30b-4f94-8b67-9b6479144183" containerID="1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07" exitCode=0
Jul 24 23:41:47.306318 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.306178    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"
Jul 24 23:41:47.306318 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.306201    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr" event={"ID":"1e84c4ca-e30b-4f94-8b67-9b6479144183","Type":"ContainerDied","Data":"1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07"}
Jul 24 23:41:47.306318 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.306239    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr" event={"ID":"1e84c4ca-e30b-4f94-8b67-9b6479144183","Type":"ContainerDied","Data":"a2677349e7f989dab2980cce110799acbd1c0168f07a572e8fa54444831f8c3d"}
Jul 24 23:41:47.306318 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.306282    2568 scope.go:117] "RemoveContainer" containerID="1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07"
Jul 24 23:41:47.315331 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.315312    2568 scope.go:117] "RemoveContainer" containerID="1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07"
Jul 24 23:41:47.315596 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:41:47.315572    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07\": container with ID starting with 1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07 not found: ID does not exist" containerID="1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07"
Jul 24 23:41:47.315670 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.315606    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07"} err="failed to get container status \"1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07\": rpc error: code = NotFound desc = could not find container \"1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07\": container with ID starting with 1da66d8d0ffd8bde5fe87c8367a0a71da5c801b33dfdc02ed58c25bb9437fa07 not found: ID does not exist"
Jul 24 23:41:47.322769 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.322748    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"]
Jul 24 23:41:47.326593 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:47.326574    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3fb847-7b95fb864d-frcbr"]
Jul 24 23:41:49.258182 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:41:49.258150    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e84c4ca-e30b-4f94-8b67-9b6479144183" path="/var/lib/kubelet/pods/1e84c4ca-e30b-4f94-8b67-9b6479144183/volumes"
Jul 24 23:42:08.853015 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:08.852982    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"]
Jul 24 23:42:08.853617 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:08.853206    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2" podUID="2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f" containerName="maas-api" containerID="cri-o://de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d" gracePeriod=30
Jul 24 23:42:08.908817 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:08.908780    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"]
Jul 24 23:42:08.909402 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:08.909040    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989" podUID="48a2fda1-34a1-4885-b67e-f1876ee6f404" containerName="payload-processing" containerID="cri-o://cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb" gracePeriod=30
Jul 24 23:42:08.911061 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:08.911036    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"]
Jul 24 23:42:08.911227 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:08.911208    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp" podUID="85be21c6-3865-4916-8c99-dc37fbacdf09" containerName="payload-pre-processing" containerID="cri-o://a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c" gracePeriod=30
Jul 24 23:42:09.161995 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.161965    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:42:09.202590 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.202568    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:42:09.208460 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.208441    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:42:09.323186 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.323158    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ghr6\" (UniqueName: \"kubernetes.io/projected/48a2fda1-34a1-4885-b67e-f1876ee6f404-kube-api-access-4ghr6\") pod \"48a2fda1-34a1-4885-b67e-f1876ee6f404\" (UID: \"48a2fda1-34a1-4885-b67e-f1876ee6f404\") "
Jul 24 23:42:09.323359 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.323214    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gt7c\" (UniqueName: \"kubernetes.io/projected/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-kube-api-access-4gt7c\") pod \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") "
Jul 24 23:42:09.323441 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.323423    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6h74\" (UniqueName: \"kubernetes.io/projected/85be21c6-3865-4916-8c99-dc37fbacdf09-kube-api-access-c6h74\") pod \"85be21c6-3865-4916-8c99-dc37fbacdf09\" (UID: \"85be21c6-3865-4916-8c99-dc37fbacdf09\") "
Jul 24 23:42:09.323478 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.323467    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48a2fda1-34a1-4885-b67e-f1876ee6f404-plugins-config-volume\") pod \"48a2fda1-34a1-4885-b67e-f1876ee6f404\" (UID: \"48a2fda1-34a1-4885-b67e-f1876ee6f404\") "
Jul 24 23:42:09.323518 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.323507    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-maas-api-tls\") pod \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") "
Jul 24 23:42:09.323563 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.323552    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85be21c6-3865-4916-8c99-dc37fbacdf09-plugins-config-volume\") pod \"85be21c6-3865-4916-8c99-dc37fbacdf09\" (UID: \"85be21c6-3865-4916-8c99-dc37fbacdf09\") "
Jul 24 23:42:09.323626 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.323576    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-openshift-service-ca\") pod \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\" (UID: \"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f\") "
Jul 24 23:42:09.324175 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.323919    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48a2fda1-34a1-4885-b67e-f1876ee6f404-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "48a2fda1-34a1-4885-b67e-f1876ee6f404" (UID: "48a2fda1-34a1-4885-b67e-f1876ee6f404"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:42:09.324175 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.323968    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85be21c6-3865-4916-8c99-dc37fbacdf09-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85be21c6-3865-4916-8c99-dc37fbacdf09" (UID: "85be21c6-3865-4916-8c99-dc37fbacdf09"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:42:09.324175 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.324125    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f" (UID: "2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:42:09.325483 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.325455    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85be21c6-3865-4916-8c99-dc37fbacdf09-kube-api-access-c6h74" (OuterVolumeSpecName: "kube-api-access-c6h74") pod "85be21c6-3865-4916-8c99-dc37fbacdf09" (UID: "85be21c6-3865-4916-8c99-dc37fbacdf09"). InnerVolumeSpecName "kube-api-access-c6h74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:42:09.325571 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.325516    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a2fda1-34a1-4885-b67e-f1876ee6f404-kube-api-access-4ghr6" (OuterVolumeSpecName: "kube-api-access-4ghr6") pod "48a2fda1-34a1-4885-b67e-f1876ee6f404" (UID: "48a2fda1-34a1-4885-b67e-f1876ee6f404"). InnerVolumeSpecName "kube-api-access-4ghr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:42:09.325731 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.325713    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-kube-api-access-4gt7c" (OuterVolumeSpecName: "kube-api-access-4gt7c") pod "2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f" (UID: "2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f"). InnerVolumeSpecName "kube-api-access-4gt7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:42:09.325792 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.325776    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f" (UID: "2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:42:09.410609 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.410539    2568 generic.go:356] "Generic (PLEG): container finished" podID="2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f" containerID="de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d" exitCode=0
Jul 24 23:42:09.410609 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.410581    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"
Jul 24 23:42:09.410775 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.410625    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2" event={"ID":"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f","Type":"ContainerDied","Data":"de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d"}
Jul 24 23:42:09.410775 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.410681    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2" event={"ID":"2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f","Type":"ContainerDied","Data":"04116c52ddb5314fafe064f5c7697afc6bbab4458132c5ef4fc995cf5e898a39"}
Jul 24 23:42:09.410775 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.410712    2568 scope.go:117] "RemoveContainer" containerID="de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d"
Jul 24 23:42:09.411928 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.411910    2568 generic.go:356] "Generic (PLEG): container finished" podID="48a2fda1-34a1-4885-b67e-f1876ee6f404" containerID="cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb" exitCode=0
Jul 24 23:42:09.412031 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.411949    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"
Jul 24 23:42:09.412031 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.411993    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989" event={"ID":"48a2fda1-34a1-4885-b67e-f1876ee6f404","Type":"ContainerDied","Data":"cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb"}
Jul 24 23:42:09.412141 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.412048    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989" event={"ID":"48a2fda1-34a1-4885-b67e-f1876ee6f404","Type":"ContainerDied","Data":"8bc30736fbadced468b00b38dd094c7438ac396c5e97ee36bfa106289fb42233"}
Jul 24 23:42:09.413424 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.413405    2568 generic.go:356] "Generic (PLEG): container finished" podID="85be21c6-3865-4916-8c99-dc37fbacdf09" containerID="a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c" exitCode=0
Jul 24 23:42:09.413550 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.413439    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp" event={"ID":"85be21c6-3865-4916-8c99-dc37fbacdf09","Type":"ContainerDied","Data":"a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c"}
Jul 24 23:42:09.413550 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.413456    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"
Jul 24 23:42:09.413550 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.413471    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp" event={"ID":"85be21c6-3865-4916-8c99-dc37fbacdf09","Type":"ContainerDied","Data":"d851e534ebc5421d4b69464a6630e49e97cc6f71bd31226312d7396a1cebf6b5"}
Jul 24 23:42:09.420624 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.420601    2568 scope.go:117] "RemoveContainer" containerID="de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d"
Jul 24 23:42:09.421033 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:42:09.420914    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d\": container with ID starting with de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d not found: ID does not exist" containerID="de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d"
Jul 24 23:42:09.421033 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.420948    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d"} err="failed to get container status \"de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d\": rpc error: code = NotFound desc = could not find container \"de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d\": container with ID starting with de2b102ca839be666af1d14f2ca246afa9e67e4c9bad27d930b85909cb19737d not found: ID does not exist"
Jul 24 23:42:09.421033 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.420969    2568 scope.go:117] "RemoveContainer" containerID="cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb"
Jul 24 23:42:09.424655 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.424612    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6h74\" (UniqueName: \"kubernetes.io/projected/85be21c6-3865-4916-8c99-dc37fbacdf09-kube-api-access-c6h74\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:09.424655 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.424633    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48a2fda1-34a1-4885-b67e-f1876ee6f404-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:09.424655 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.424649    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:09.424820 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.424662    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85be21c6-3865-4916-8c99-dc37fbacdf09-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:09.424820 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.424676    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:09.424820 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.424689    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ghr6\" (UniqueName: \"kubernetes.io/projected/48a2fda1-34a1-4885-b67e-f1876ee6f404-kube-api-access-4ghr6\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:09.424820 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.424703    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gt7c\" (UniqueName: \"kubernetes.io/projected/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f-kube-api-access-4gt7c\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:09.429173 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.429155    2568 scope.go:117] "RemoveContainer" containerID="cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb"
Jul 24 23:42:09.429441 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:42:09.429421    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb\": container with ID starting with cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb not found: ID does not exist" containerID="cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb"
Jul 24 23:42:09.429511 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.429446    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb"} err="failed to get container status \"cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb\": rpc error: code = NotFound desc = could not find container \"cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb\": container with ID starting with cf75dbb5a5b01c6b44e30ac1a88c4354e0f46b7935a990cb3e8d51c3e916bfbb not found: ID does not exist"
Jul 24 23:42:09.429511 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.429463    2568 scope.go:117] "RemoveContainer" containerID="a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c"
Jul 24 23:42:09.436646 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.436622    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"]
Jul 24 23:42:09.436934 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.436920    2568 scope.go:117] "RemoveContainer" containerID="a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c"
Jul 24 23:42:09.437188 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:42:09.437173    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c\": container with ID starting with a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c not found: ID does not exist" containerID="a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c"
Jul 24 23:42:09.437270 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.437204    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c"} err="failed to get container status \"a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c\": rpc error: code = NotFound desc = could not find container \"a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c\": container with ID starting with a8c2a24de5c4934268b15cff850e1d2aba1e213b1764b310569bc686a62bb45c not found: ID does not exist"
Jul 24 23:42:09.440381 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.440359    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fe475f-75f9d77976-hbhb2"]
Jul 24 23:42:09.450196 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.450166    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"]
Jul 24 23:42:09.454007 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.453980    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fe475f-646799dc8b-kz989"]
Jul 24 23:42:09.464126 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.464103    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"]
Jul 24 23:42:09.468064 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:09.468046    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fe475f-7f686f9948-s2nqp"]
Jul 24 23:42:11.263038 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:11.262998    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f" path="/var/lib/kubelet/pods/2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f/volumes"
Jul 24 23:42:11.263515 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:11.263498    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48a2fda1-34a1-4885-b67e-f1876ee6f404" path="/var/lib/kubelet/pods/48a2fda1-34a1-4885-b67e-f1876ee6f404/volumes"
Jul 24 23:42:11.263965 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:11.263948    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85be21c6-3865-4916-8c99-dc37fbacdf09" path="/var/lib/kubelet/pods/85be21c6-3865-4916-8c99-dc37fbacdf09/volumes"
Jul 24 23:42:30.470033 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.470001    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"]
Jul 24 23:42:30.470547 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.470190    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s" podUID="5a19449e-401c-4edb-822b-795e502e6726" containerName="maas-api" containerID="cri-o://48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301" gracePeriod=30
Jul 24 23:42:30.523883 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.523852    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"]
Jul 24 23:42:30.524086 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.524064    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf" podUID="b9451d15-f546-40ac-8b94-2633b7a7e1fd" containerName="payload-processing" containerID="cri-o://28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30" gracePeriod=30
Jul 24 23:42:30.768696 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.768674    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:42:30.813550 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.813522    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:42:30.824098 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.824074    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a19449e-401c-4edb-822b-795e502e6726-openshift-service-ca\") pod \"5a19449e-401c-4edb-822b-795e502e6726\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") "
Jul 24 23:42:30.824260 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.824141    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a19449e-401c-4edb-822b-795e502e6726-maas-api-tls\") pod \"5a19449e-401c-4edb-822b-795e502e6726\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") "
Jul 24 23:42:30.824260 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.824204    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4c2j\" (UniqueName: \"kubernetes.io/projected/5a19449e-401c-4edb-822b-795e502e6726-kube-api-access-g4c2j\") pod \"5a19449e-401c-4edb-822b-795e502e6726\" (UID: \"5a19449e-401c-4edb-822b-795e502e6726\") "
Jul 24 23:42:30.824459 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.824429    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a19449e-401c-4edb-822b-795e502e6726-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a19449e-401c-4edb-822b-795e502e6726" (UID: "5a19449e-401c-4edb-822b-795e502e6726"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:42:30.826187 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.826166    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a19449e-401c-4edb-822b-795e502e6726-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a19449e-401c-4edb-822b-795e502e6726" (UID: "5a19449e-401c-4edb-822b-795e502e6726"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 23:42:30.826299 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.826213    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a19449e-401c-4edb-822b-795e502e6726-kube-api-access-g4c2j" (OuterVolumeSpecName: "kube-api-access-g4c2j") pod "5a19449e-401c-4edb-822b-795e502e6726" (UID: "5a19449e-401c-4edb-822b-795e502e6726"). InnerVolumeSpecName "kube-api-access-g4c2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:42:30.924873 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.924840    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gqpt\" (UniqueName: \"kubernetes.io/projected/b9451d15-f546-40ac-8b94-2633b7a7e1fd-kube-api-access-4gqpt\") pod \"b9451d15-f546-40ac-8b94-2633b7a7e1fd\" (UID: \"b9451d15-f546-40ac-8b94-2633b7a7e1fd\") "
Jul 24 23:42:30.925046 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.924985    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9451d15-f546-40ac-8b94-2633b7a7e1fd-plugins-config-volume\") pod \"b9451d15-f546-40ac-8b94-2633b7a7e1fd\" (UID: \"b9451d15-f546-40ac-8b94-2633b7a7e1fd\") "
Jul 24 23:42:30.925232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.925205    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a19449e-401c-4edb-822b-795e502e6726-openshift-service-ca\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:30.925232 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.925229    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a19449e-401c-4edb-822b-795e502e6726-maas-api-tls\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:30.925419 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.925245    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4c2j\" (UniqueName: \"kubernetes.io/projected/5a19449e-401c-4edb-822b-795e502e6726-kube-api-access-g4c2j\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:30.925419 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.925381    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9451d15-f546-40ac-8b94-2633b7a7e1fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b9451d15-f546-40ac-8b94-2633b7a7e1fd" (UID: "b9451d15-f546-40ac-8b94-2633b7a7e1fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 23:42:30.926855 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:30.926833    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9451d15-f546-40ac-8b94-2633b7a7e1fd-kube-api-access-4gqpt" (OuterVolumeSpecName: "kube-api-access-4gqpt") pod "b9451d15-f546-40ac-8b94-2633b7a7e1fd" (UID: "b9451d15-f546-40ac-8b94-2633b7a7e1fd"). InnerVolumeSpecName "kube-api-access-4gqpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 23:42:31.026244 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.026167    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gqpt\" (UniqueName: \"kubernetes.io/projected/b9451d15-f546-40ac-8b94-2633b7a7e1fd-kube-api-access-4gqpt\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:31.026244 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.026197    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9451d15-f546-40ac-8b94-2633b7a7e1fd-plugins-config-volume\") on node \"ip-10-0-132-153.ec2.internal\" DevicePath \"\""
Jul 24 23:42:31.516455 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.516420    2568 generic.go:356] "Generic (PLEG): container finished" podID="5a19449e-401c-4edb-822b-795e502e6726" containerID="48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301" exitCode=0
Jul 24 23:42:31.516891 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.516479    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"
Jul 24 23:42:31.516891 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.516502    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s" event={"ID":"5a19449e-401c-4edb-822b-795e502e6726","Type":"ContainerDied","Data":"48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301"}
Jul 24 23:42:31.516891 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.516552    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s" event={"ID":"5a19449e-401c-4edb-822b-795e502e6726","Type":"ContainerDied","Data":"c13c612e98b7bd9f2d72af194bc657fe4ad3e57e13d8738bf68c66f4538b8231"}
Jul 24 23:42:31.516891 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.516582    2568 scope.go:117] "RemoveContainer" containerID="48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301"
Jul 24 23:42:31.517938 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.517920    2568 generic.go:356] "Generic (PLEG): container finished" podID="b9451d15-f546-40ac-8b94-2633b7a7e1fd" containerID="28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30" exitCode=0
Jul 24 23:42:31.518015 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.517963    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"
Jul 24 23:42:31.518060 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.517963    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf" event={"ID":"b9451d15-f546-40ac-8b94-2633b7a7e1fd","Type":"ContainerDied","Data":"28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30"}
Jul 24 23:42:31.518099 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.518086    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf" event={"ID":"b9451d15-f546-40ac-8b94-2633b7a7e1fd","Type":"ContainerDied","Data":"e99f610194572bf25b3a82aec8d2f7ee2850abf9d0e01b101ce4bfb97fb0562a"}
Jul 24 23:42:31.525306 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.525284    2568 scope.go:117] "RemoveContainer" containerID="48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301"
Jul 24 23:42:31.525561 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:42:31.525544    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301\": container with ID starting with 48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301 not found: ID does not exist" containerID="48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301"
Jul 24 23:42:31.525640 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.525572    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301"} err="failed to get container status \"48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301\": rpc error: code = NotFound desc = could not find container \"48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301\": container with ID starting with 48e0ff824e93a6dcc9daf047ee7775d5916ffa3959aa7baa26fac74678b62301 not found: ID does not exist"
Jul 24 23:42:31.525640 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.525597    2568 scope.go:117] "RemoveContainer" containerID="28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30"
Jul 24 23:42:31.532770 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.532752    2568 scope.go:117] "RemoveContainer" containerID="28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30"
Jul 24 23:42:31.533087 ip-10-0-132-153 kubenswrapper[2568]: E0724 23:42:31.533070    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30\": container with ID starting with 28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30 not found: ID does not exist" containerID="28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30"
Jul 24 23:42:31.533157 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.533091    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30"} err="failed to get container status \"28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30\": rpc error: code = NotFound desc = could not find container \"28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30\": container with ID starting with 28f57146bd7f0a1b0c335b996f0ddb75a9a42d3b362c8ec366d863c545421b30 not found: ID does not exist"
Jul 24 23:42:31.541660 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.541638    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"]
Jul 24 23:42:31.544539 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.544515    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-6f3ce2-96f9ff4f-ndr6s"]
Jul 24 23:42:31.553812 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.553788    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"]
Jul 24 23:42:31.555948 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:31.555928    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6f3ce2-5747dbbcc4-d7gbf"]
Jul 24 23:42:33.258076 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:33.258045    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a19449e-401c-4edb-822b-795e502e6726" path="/var/lib/kubelet/pods/5a19449e-401c-4edb-822b-795e502e6726/volumes"
Jul 24 23:42:33.258481 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:33.258438    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9451d15-f546-40ac-8b94-2633b7a7e1fd" path="/var/lib/kubelet/pods/b9451d15-f546-40ac-8b94-2633b7a7e1fd/volumes"
Jul 24 23:42:48.267145 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:48.267115    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777c88b5cf-q5lhr_26157995-be18-49f3-b427-2428485abb49/copy-manifests/0.log"
Jul 24 23:42:48.274134 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:48.274112    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777c88b5cf-q5lhr_26157995-be18-49f3-b427-2428485abb49/manager/0.log"
Jul 24 23:42:48.738137 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:48.738107    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7d9754897c-lg5pg_8fd80415-b33f-4ced-9273-f590032afcc1/maas-api/0.log"
Jul 24 23:42:48.962324 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:48.962295    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748930-jj68s_0c592212-e1f3-4a70-9ad6-d15da3e01353/cleanup/0.log"
Jul 24 23:42:49.074950 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:49.074861    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-c8l4w_e9d4f6f6-a9e8-468d-8b8f-d52264498b3d/postgres/0.log"
Jul 24 23:42:50.728901 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:50.728876    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-g2tfk_4c931f7c-6c3c-4ff4-b7ab-278ef80f0798/kuadrant-console-plugin/0.log"
Jul 24 23:42:51.824505 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:51.824480    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745f8948c6-6sd86_3361a14a-c83a-4060-8107-33c16983c567/kube-auth-proxy/0.log"
Jul 24 23:42:52.864395 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:52.864365    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vst4v_24636d55-feb3-4eaf-9cfb-86312f4e8e70/main/0.log"
Jul 24 23:42:52.980874 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:52.980848    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5kbfr_39d98e22-18bc-4df2-a552-21471d2a3ba7/main/0.log"
Jul 24 23:42:53.318422 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:42:53.318395    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-85ldr_0daa5fc8-079b-4c42-a702-b3de326ff7a2/main/0.log"
Jul 24 23:43:01.609945 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:01.609911    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cm8cs_105e164b-d539-4eb1-83e1-0db3b18886ca/global-pull-secret-syncer/0.log"
Jul 24 23:43:01.788408 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:01.788378    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wvrhp_a9b51fe6-7dcd-4f8f-9a8f-4efe2d3a7cc3/konnectivity-agent/0.log"
Jul 24 23:43:01.814491 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:01.814463    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-153.ec2.internal_e2c72bef2cb1e0ad183f8af7f40f58d5/haproxy/0.log"
Jul 24 23:43:07.137134 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:07.137103    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-g2tfk_4c931f7c-6c3c-4ff4-b7ab-278ef80f0798/kuadrant-console-plugin/0.log"
Jul 24 23:43:09.648144 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:09.648114    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7cfb7c5c67-6b7wk_03ba85c6-1163-4ee1-aa37-ac3c6e3493b5/metrics-server/0.log"
Jul 24 23:43:09.672110 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:09.672083    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-fw6st_02a665f6-1d07-498a-b1ed-c91e6ba7700d/monitoring-plugin/0.log"
Jul 24 23:43:09.706351 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:09.706329    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fxzgx_e40a84ee-b980-43f4-9c9a-1b181e5c8519/node-exporter/0.log"
Jul 24 23:43:09.726932 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:09.726912    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fxzgx_e40a84ee-b980-43f4-9c9a-1b181e5c8519/kube-rbac-proxy/0.log"
Jul 24 23:43:09.749305 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:09.749279    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fxzgx_e40a84ee-b980-43f4-9c9a-1b181e5c8519/init-textfile/0.log"
Jul 24 23:43:10.048054 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.048032    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2c442b-2c28-4a45-bea5-825c68803d0c/prometheus/0.log"
Jul 24 23:43:10.077184 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.077165    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2c442b-2c28-4a45-bea5-825c68803d0c/config-reloader/0.log"
Jul 24 23:43:10.103492 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.103473    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2c442b-2c28-4a45-bea5-825c68803d0c/thanos-sidecar/0.log"
Jul 24 23:43:10.131200 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.131175    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2c442b-2c28-4a45-bea5-825c68803d0c/kube-rbac-proxy-web/0.log"
Jul 24 23:43:10.158381 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.158363    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2c442b-2c28-4a45-bea5-825c68803d0c/kube-rbac-proxy/0.log"
Jul 24 23:43:10.184354 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.184335    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2c442b-2c28-4a45-bea5-825c68803d0c/kube-rbac-proxy-thanos/0.log"
Jul 24 23:43:10.214597 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.214577    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2c442b-2c28-4a45-bea5-825c68803d0c/init-config-reloader/0.log"
Jul 24 23:43:10.244436 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.244409    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-jvnzt_9a121134-ab85-45c1-bf5c-ed6c849e02fc/prometheus-operator/0.log"
Jul 24 23:43:10.266077 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.266057    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-jvnzt_9a121134-ab85-45c1-bf5c-ed6c849e02fc/kube-rbac-proxy/0.log"
Jul 24 23:43:10.325030 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.324975    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7465458b77-pxq84_d4e4cd8b-42a7-4054-8b07-8ed4a091db70/telemeter-client/0.log"
Jul 24 23:43:10.349386 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.349369    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7465458b77-pxq84_d4e4cd8b-42a7-4054-8b07-8ed4a091db70/reload/0.log"
Jul 24 23:43:10.370891 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.370869    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7465458b77-pxq84_d4e4cd8b-42a7-4054-8b07-8ed4a091db70/kube-rbac-proxy/0.log"
Jul 24 23:43:10.404948 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.404929    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64888c8bd7-6qvn2_3fdad54c-a9ea-4f27-a809-06663d90eab0/thanos-query/0.log"
Jul 24 23:43:10.435013 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.434997    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64888c8bd7-6qvn2_3fdad54c-a9ea-4f27-a809-06663d90eab0/kube-rbac-proxy-web/0.log"
Jul 24 23:43:10.456814 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.456797    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64888c8bd7-6qvn2_3fdad54c-a9ea-4f27-a809-06663d90eab0/kube-rbac-proxy/0.log"
Jul 24 23:43:10.478861 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.478841    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64888c8bd7-6qvn2_3fdad54c-a9ea-4f27-a809-06663d90eab0/prom-label-proxy/0.log"
Jul 24 23:43:10.503222 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.503205    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64888c8bd7-6qvn2_3fdad54c-a9ea-4f27-a809-06663d90eab0/kube-rbac-proxy-rules/0.log"
Jul 24 23:43:10.525999 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:10.525979    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64888c8bd7-6qvn2_3fdad54c-a9ea-4f27-a809-06663d90eab0/kube-rbac-proxy-metrics/0.log"
Jul 24 23:43:11.798035 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798002    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"]
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798524    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9451d15-f546-40ac-8b94-2633b7a7e1fd" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798545    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9451d15-f546-40ac-8b94-2633b7a7e1fd" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798567    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b95b46d-8d01-4e7c-a187-44322a6a7453" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798575    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b95b46d-8d01-4e7c-a187-44322a6a7453" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798589    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5100dd3-83a6-4ee0-8402-e35a09f5d208" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798598    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5100dd3-83a6-4ee0-8402-e35a09f5d208" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798619    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48a2fda1-34a1-4885-b67e-f1876ee6f404" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798626    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="48a2fda1-34a1-4885-b67e-f1876ee6f404" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798637    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85be21c6-3865-4916-8c99-dc37fbacdf09" containerName="payload-pre-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798645    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="85be21c6-3865-4916-8c99-dc37fbacdf09" containerName="payload-pre-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798653    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="871c48ab-e84b-46fc-8f94-9f52ed81b99e" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798661    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="871c48ab-e84b-46fc-8f94-9f52ed81b99e" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798668    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e84c4ca-e30b-4f94-8b67-9b6479144183" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798675    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e84c4ca-e30b-4f94-8b67-9b6479144183" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798685    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a19449e-401c-4edb-822b-795e502e6726" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798693    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a19449e-401c-4edb-822b-795e502e6726" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798706    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b68d9682-be9f-4612-8603-bc549c38e85e" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798714    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b68d9682-be9f-4612-8603-bc549c38e85e" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798726    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798734    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798813    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="871c48ab-e84b-46fc-8f94-9f52ed81b99e" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798825    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b95b46d-8d01-4e7c-a187-44322a6a7453" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798833    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9451d15-f546-40ac-8b94-2633b7a7e1fd" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798841    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="48a2fda1-34a1-4885-b67e-f1876ee6f404" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798849    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="85be21c6-3865-4916-8c99-dc37fbacdf09" containerName="payload-pre-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798857    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5100dd3-83a6-4ee0-8402-e35a09f5d208" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798865    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cfb6d7f-53ea-458a-b0f9-dbdd8562e98f" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798874    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b68d9682-be9f-4612-8603-bc549c38e85e" containerName="payload-processing"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798883    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a19449e-401c-4edb-822b-795e502e6726" containerName="maas-api"
Jul 24 23:43:11.799017 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.798895    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e84c4ca-e30b-4f94-8b67-9b6479144183" containerName="payload-processing"
Jul 24 23:43:11.802107 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.802092    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.805286 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.805258    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jpvfn\"/\"openshift-service-ca.crt\""
Jul 24 23:43:11.806209 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.806191    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jpvfn\"/\"default-dockercfg-br4hs\""
Jul 24 23:43:11.806339 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.806219    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jpvfn\"/\"kube-root-ca.crt\""
Jul 24 23:43:11.812860 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.812828    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"]
Jul 24 23:43:11.885153 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.885125    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxv84\" (UniqueName: \"kubernetes.io/projected/c79e6a57-037c-42d6-8ede-57041e050087-kube-api-access-cxv84\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.885318 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.885221    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-lib-modules\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.885318 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.885274    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-podres\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.885318 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.885292    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-proc\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.885318 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.885310    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-sys\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.985727 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.985698    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-lib-modules\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.985884 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.985739    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-podres\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.985884 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.985757    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-proc\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.985884 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.985773    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-sys\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.985884 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.985798    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxv84\" (UniqueName: \"kubernetes.io/projected/c79e6a57-037c-42d6-8ede-57041e050087-kube-api-access-cxv84\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.985884 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.985850    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-proc\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.985884 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.985860    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-sys\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.985884 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.985864    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-podres\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.985884 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.985880    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c79e6a57-037c-42d6-8ede-57041e050087-lib-modules\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:11.994817 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:11.994796    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxv84\" (UniqueName: \"kubernetes.io/projected/c79e6a57-037c-42d6-8ede-57041e050087-kube-api-access-cxv84\") pod \"perf-node-gather-daemonset-qn2x2\" (UID: \"c79e6a57-037c-42d6-8ede-57041e050087\") " pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:12.111720 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:12.111600    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:12.243404 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:12.243378    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"]
Jul 24 23:43:12.245385 ip-10-0-132-153 kubenswrapper[2568]: W0724 23:43:12.245358    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc79e6a57_037c_42d6_8ede_57041e050087.slice/crio-5ab3c42c34a8afe99d162b3c4226e88c91b6892e38db398f94b5750b93cf92f7 WatchSource:0}: Error finding container 5ab3c42c34a8afe99d162b3c4226e88c91b6892e38db398f94b5750b93cf92f7: Status 404 returned error can't find the container with id 5ab3c42c34a8afe99d162b3c4226e88c91b6892e38db398f94b5750b93cf92f7
Jul 24 23:43:12.706871 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:12.706836    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2" event={"ID":"c79e6a57-037c-42d6-8ede-57041e050087","Type":"ContainerStarted","Data":"e9bbefeac6c68f9efe8fa0593a96ec4abb0a4c7959e3ec87aad94580dd7dcfe3"}
Jul 24 23:43:12.707091 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:12.706886    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2" event={"ID":"c79e6a57-037c-42d6-8ede-57041e050087","Type":"ContainerStarted","Data":"5ab3c42c34a8afe99d162b3c4226e88c91b6892e38db398f94b5750b93cf92f7"}
Jul 24 23:43:12.707091 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:12.706946    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:12.724863 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:12.724818    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2" podStartSLOduration=1.724803041 podStartE2EDuration="1.724803041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:43:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:43:12.723570788 +0000 UTC m=+3050.000604332" watchObservedRunningTime="2026-07-24 23:43:12.724803041 +0000 UTC m=+3050.001836585"
Jul 24 23:43:14.677607 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:14.677537    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cfwc9_f157f026-514d-4b81-8497-6b39a07b9c58/dns/0.log"
Jul 24 23:43:14.698564 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:14.698541    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cfwc9_f157f026-514d-4b81-8497-6b39a07b9c58/kube-rbac-proxy/0.log"
Jul 24 23:43:14.860930 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:14.860899    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xllwh_fac3b1fc-9561-4722-9701-abb84981e59b/dns-node-resolver/0.log"
Jul 24 23:43:15.419363 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:15.419335    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-595bf88bc6-dj7ql_5a2b5a24-16bd-4d43-b0e0-6bb5c117359e/registry/0.log"
Jul 24 23:43:15.486468 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:15.486439    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lxdcd_64863c56-9777-45d6-8502-328507a7421e/node-ca/0.log"
Jul 24 23:43:16.703875 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:16.703843    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745f8948c6-6sd86_3361a14a-c83a-4060-8107-33c16983c567/kube-auth-proxy/0.log"
Jul 24 23:43:17.761093 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:17.761059    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ggkzp_08c1379f-90b8-41f6-b19c-1bb382937934/serve-healthcheck-canary/0.log"
Jul 24 23:43:18.254905 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:18.254878    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-m78lj_9e7719c2-57e4-426d-8aee-742e2e535b19/insights-operator/0.log"
Jul 24 23:43:18.255835 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:18.255811    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-m78lj_9e7719c2-57e4-426d-8aee-742e2e535b19/insights-operator/1.log"
Jul 24 23:43:18.419014 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:18.418986    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqdf5_17c261bd-15e5-4c8f-9cf6-8cab547c1c80/kube-rbac-proxy/0.log"
Jul 24 23:43:18.442174 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:18.442151    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqdf5_17c261bd-15e5-4c8f-9cf6-8cab547c1c80/exporter/0.log"
Jul 24 23:43:18.463562 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:18.463541    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqdf5_17c261bd-15e5-4c8f-9cf6-8cab547c1c80/extractor/0.log"
Jul 24 23:43:18.721415 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:18.721393    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jpvfn/perf-node-gather-daemonset-qn2x2"
Jul 24 23:43:21.762534 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:21.762511    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777c88b5cf-q5lhr_26157995-be18-49f3-b427-2428485abb49/manager/0.log"
Jul 24 23:43:21.786040 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:21.786019    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777c88b5cf-q5lhr_26157995-be18-49f3-b427-2428485abb49/copy-manifests/0.log"
Jul 24 23:43:23.081049 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:23.081024    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6bccc7ffcc-s4t5m_da543487-63fd-4dc0-8303-bedd8496c164/manager/0.log"
Jul 24 23:43:23.163296 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:23.163264    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745f8948c6-6sd86_3361a14a-c83a-4060-8107-33c16983c567/kube-auth-proxy/0.log"
Jul 24 23:43:24.981196 ip-10-0-132-153 kubenswrapper[2568]: I0724 23:43:24.981171    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745f8948c6-6sd86_3361a14a-c83a-4060-8107-33c16983c567/kube-auth-proxy/0.log"