Jul 26 15:38:04.368171 ip-10-0-132-10 systemd[1]: Starting Kubernetes Kubelet...
Jul 26 15:38:04.816686 ip-10-0-132-10 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 15:38:04.816686 ip-10-0-132-10 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 26 15:38:04.816686 ip-10-0-132-10 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 15:38:04.816686 ip-10-0-132-10 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 26 15:38:04.816686 ip-10-0-132-10 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 15:38:04.817296 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.816759    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 26 15:38:04.822361 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822347    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:04.822361 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822360    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822366    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822370    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822373    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822377    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822380    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822383    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822385    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822388    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822391    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822393    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822396    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822404    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822407    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822409    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822412    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822414    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822417    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822419    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822421    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:04.822424 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822424    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822426    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822429    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822432    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822434    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822437    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822440    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822442    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822445    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822448    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822450    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822452    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822455    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822458    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822462    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822464    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822467    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822469    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822472    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:04.822926 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822474    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822476    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822479    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822481    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822484    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822486    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822489    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822491    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822493    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822496    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822498    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822500    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822503    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822506    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822508    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822511    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822515    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822518    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822521    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:04.823370 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822523    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822526    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822529    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822531    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822534    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822537    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822539    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822542    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822544    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822547    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822549    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822553    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822555    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822558    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822560    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822562    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822565    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822568    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822571    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822573    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:04.823817 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822576    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822579    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822581    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822584    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822586    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822588    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822591    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822593    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822597    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822599    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822601    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822604    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822607    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822609    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822611    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822614    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822616    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822619    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822621    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.822624    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:04.824295 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824126    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824133    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824136    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824139    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824143    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824147    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824150    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824152    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824155    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824158    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824161    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824163    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824165    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824169    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824173    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824176    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824179    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824181    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824184    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:04.824751 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824187    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824190    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824194    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824196    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824199    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824202    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824204    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824207    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824209    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824211    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824214    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824216    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824219    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824221    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824224    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824226    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824228    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824232    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824234    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824237    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:04.825326 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824239    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824241    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824244    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824246    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824248    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824251    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824253    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824256    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824259    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824261    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824263    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824266    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824268    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824271    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824274    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824277    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824279    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824281    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824284    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824286    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:04.825808 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824289    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824291    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824294    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824296    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824300    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824303    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824306    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824308    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824311    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824313    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824316    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824320    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824323    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824326    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824329    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824331    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824334    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824336    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824338    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:04.826298 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824341    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824344    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824346    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824349    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824351    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824354    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824357    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824360    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824362    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824365    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824367    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824370    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824372    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824374    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824377    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824379    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824382    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824384    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824386    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824389    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:04.826752 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824391    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824720    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824724    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824727    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824729    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824732    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824734    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824737    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824740    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824742    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824745    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824748    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824751    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824754    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824757    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824759    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824762    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824765    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824767    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824770    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:04.827236 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824772    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824775    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824777    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824780    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824782    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824784    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824787    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824789    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824791    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824794    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824796    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824799    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824801    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824804    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824806    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824808    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824812    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824816    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:04.827695 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824820    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824823    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824825    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824829    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824831    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824834    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824837    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824840    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824842    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824845    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824847    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824850    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824853    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824855    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824858    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824860    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824862    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824865    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824868    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:04.828145 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824871    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824874    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824876    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824879    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824881    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824883    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824885    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824901    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824904    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824906    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824910    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824912    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824914    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824917    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824919    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824922    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824925    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824928    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824930    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824933    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:04.828594 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824935    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824938    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824940    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824942    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824945    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824947    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824950    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824952    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824954    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824957    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824959    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824961    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824964    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824966    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824968    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824971    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824973    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824975    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824977    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824980    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:04.829074 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824989    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824991    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.824994    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825056    2560 flags.go:64] FLAG: --address="0.0.0.0"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825063    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825068    2560 flags.go:64] FLAG: --anonymous-auth="true"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825072    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825076    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825079    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825083    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825087    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825090    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825093    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825096    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825099    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825102    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825105    2560 flags.go:64] FLAG: --cgroup-root=""
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825107    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825110    2560 flags.go:64] FLAG: --client-ca-file=""
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825113    2560 flags.go:64] FLAG: --cloud-provider="external"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825116    2560 flags.go:64] FLAG: --cluster-dns="[]"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825120    2560 flags.go:64] FLAG: --cluster-domain=""
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825123    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825126    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 26 15:38:04.829541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825129    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825132    2560 flags.go:64] FLAG: --container-log-max-files="5"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825136    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825139    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825141    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825144    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825147    2560 flags.go:64] FLAG: --contention-profiling="false"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825150    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825153    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825156    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825160    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825164    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825167    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825170    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825173    2560 flags.go:64] FLAG: --enable-load-reader="false"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825176    2560 flags.go:64] FLAG: --enable-server="true"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825179    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825183    2560 flags.go:64] FLAG: --event-burst="100"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825186    2560 flags.go:64] FLAG: --event-qps="50"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825188    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825191    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825194    2560 flags.go:64] FLAG: --eviction-hard=""
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825197    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825200    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825203    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 26 15:38:04.830129 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825205    2560 flags.go:64] FLAG: --eviction-soft=""
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825208    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825211    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825213    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825216    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825219    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825221    2560 flags.go:64] FLAG: --fail-swap-on="true"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825224    2560 flags.go:64] FLAG: --feature-gates=""
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825227    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825230    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825233    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825236    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825239    2560 flags.go:64] FLAG: --healthz-port="10248"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825241    2560 flags.go:64] FLAG: --help="false"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825244    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-132-10.ec2.internal"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825247    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825250    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825255    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825259    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825262    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825265    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825268    2560 flags.go:64] FLAG: --image-service-endpoint=""
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825270    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 26 15:38:04.830727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825273    2560 flags.go:64] FLAG: --kube-api-burst="100"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825276    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825279    2560 flags.go:64] FLAG: --kube-api-qps="50"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825282    2560 flags.go:64] FLAG: --kube-reserved=""
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825285    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825287    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825290    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825293    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825296    2560 flags.go:64] FLAG: --lock-file=""
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825298    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825301    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825304    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825309    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825311    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825314    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825317    2560 flags.go:64] FLAG: --logging-format="text"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825320    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825323    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825325    2560 flags.go:64] FLAG: --manifest-url=""
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825328    2560 flags.go:64] FLAG: --manifest-url-header=""
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825335    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825338    2560 flags.go:64] FLAG: --max-open-files="1000000"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825341    2560 flags.go:64] FLAG: --max-pods="110"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825344    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825347    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 26 15:38:04.831306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825350    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825354    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825357    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825360    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825363    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825369    2560 flags.go:64] FLAG: --node-status-max-images="50"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825372    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825375    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825378    2560 flags.go:64] FLAG: --pod-cidr=""
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825381    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825386    2560 flags.go:64] FLAG: --pod-manifest-path=""
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825389    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825391    2560 flags.go:64] FLAG: --pods-per-core="0"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825394    2560 flags.go:64] FLAG: --port="10250"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825397    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825400    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06e8d4cb43070e9ac"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825403    2560 flags.go:64] FLAG: --qos-reserved=""
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825405    2560 flags.go:64] FLAG: --read-only-port="10255"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825408    2560 flags.go:64] FLAG: --register-node="true"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825411    2560 flags.go:64] FLAG: --register-with-taints=""
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825414    2560 flags.go:64] FLAG: --registry-burst="10"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825417    2560 flags.go:64] FLAG: --registry-qps="5"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825419    2560 flags.go:64] FLAG: --reserved-cpus=""
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825422    2560 flags.go:64] FLAG: --reserved-memory=""
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825425    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 26 15:38:04.831953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825428    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825431    2560 flags.go:64] FLAG: --rotate-certificates="false"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825434    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825436    2560 flags.go:64] FLAG: --runonce="false"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825439    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825441    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825444    2560 flags.go:64] FLAG: --seccomp-default="false"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825447    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825449    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825454    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825458    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825460    2560 flags.go:64] FLAG: --storage-driver-password="root"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825465    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825467    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825470    2560 flags.go:64] FLAG: --storage-driver-user="root"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825473    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825476    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825479    2560 flags.go:64] FLAG: --system-cgroups=""
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825482    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825486    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825489    2560 flags.go:64] FLAG: --tls-cert-file=""
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825492    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825496    2560 flags.go:64] FLAG: --tls-min-version=""
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825499    2560 flags.go:64] FLAG: --tls-private-key-file=""
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825501    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 26 15:38:04.832547 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825517    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825520    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825524    2560 flags.go:64] FLAG: --v="2"
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825527    2560 flags.go:64] FLAG: --version="false"
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825531    2560 flags.go:64] FLAG: --vmodule=""
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825535    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.825538    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825639    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825642    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825645    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825648    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825652    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825656    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825659    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825662    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825665    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825668    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825671    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825674    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825678    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825681    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:04.833310 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825683    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825686    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825688    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825691    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825693    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825696    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825698    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825701    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825703    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825707    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825710    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825713    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825715    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825718    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825721    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825724    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825726    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825729    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825731    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825734    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:04.834046 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825736    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825739    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825741    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825744    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825746    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825749    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825751    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825755    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825758    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825760    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825766    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825769    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825772    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825775    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825778    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825780    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825783    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825785    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825788    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:04.834599 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825790    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825793    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825795    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825798    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825800    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825802    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825805    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825807    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825810    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825813    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825815    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825818    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825821    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825823    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825826    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825828    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825831    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825833    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825835    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825838    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:04.835077 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825841    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825844    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825846    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825850    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825853    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825855    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825858    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825861    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825863    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825866    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825868    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825871    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825873    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825875    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825878    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825880    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825883    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825885    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825901    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:04.835566 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825904    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825906    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825909    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825911    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825914    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825917    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.825919    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.826933    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.834472    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.834486    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.834505    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.834514    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834565    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834570    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834574    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:04.836053 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834578    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834580    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834583    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834586    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834589    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834592    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834595    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834597    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834600    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834603    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834606    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834608    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834611    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834614    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834616    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834619    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834623    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834626    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834634    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834638    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:04.836408 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834640    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834643    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834645    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834648    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834650    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834653    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834657    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834659    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834662    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834665    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834668    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834670    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834673    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834676    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834678    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834682    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834685    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834689    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834691    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:04.836871 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834694    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834696    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834699    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834701    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834710    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834713    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834716    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834719    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834721    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834723    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834726    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834728    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834731    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834733    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834736    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834738    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834741    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834743    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834746    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834749    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:04.837335 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834752    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834754    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834757    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834760    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834762    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834764    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834767    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834770    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834772    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834775    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834777    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834787    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834790    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834792    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834795    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834797    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834800    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834808    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834811    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834814    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:04.837823 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834816    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834820    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834824    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834827    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834829    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834832    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834835    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834837    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834840    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834842    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834845    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834847    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834850    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834853    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834855    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834858    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:04.838334 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.834860    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.834865    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835005    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835011    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835014    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835016    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835019    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835021    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835024    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835026    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835029    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835031    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835034    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835037    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835046    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:04.838707 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835049    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835051    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835054    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835056    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835059    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835061    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835064    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835066    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835069    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835071    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835074    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835076    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835079    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835082    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835085    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835087    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835090    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835093    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835095    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835097    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:04.839083 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835100    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835102    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835105    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835107    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835110    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835113    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835115    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835117    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835120    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835122    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835125    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835127    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835135    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835138    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835140    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835142    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835145    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835147    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835150    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835153    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:04.839557 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835155    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835157    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835160    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835162    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835165    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835167    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835170    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835172    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835175    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835177    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835180    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835183    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835186    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835190    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835192    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835195    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835198    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835200    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835202    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:04.840045 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835205    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835207    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835211    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835214    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835217    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835219    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835228    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835230    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835232    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835235    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835238    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835240    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835243    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835245    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835248    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835250    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835253    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835255    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835257    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:04.840492 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835260    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835264    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835267    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835270    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835273    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835275    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835278    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:04.835281    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.835286    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.835390    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.837516    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.839217    2560 server.go:1019] "Starting client certificate rotation"
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.839311    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 15:38:04.840943 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.839348    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 15:38:04.842273 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.842260    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 15:38:04.843468 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.843455    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 15:38:04.851919 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.851886    2560 log.go:25] "Validated CRI v1 runtime API"
Jul 26 15:38:04.853748 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.853731    2560 log.go:25] "Validated CRI v1 image API"
Jul 26 15:38:04.854565 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.854551    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 26 15:38:04.857552 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.857535    2560 fs.go:135] Filesystem UUIDs: map[77dac6ff-54f4-4322-8972-aeebd51ebd54:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fc9d6781-7dd5-465a-8c26-348b23af3782:/dev/nvme0n1p4]
Jul 26 15:38:04.857604 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.857551    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 26 15:38:04.863852 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.863739    2560 manager.go:216] Machine: {Timestamp:2026-07-26 15:38:04.861176872 +0000 UTC m=+0.360566699 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3193761 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2de58a8834a46fedb58dbd162ed3bb SystemUUID:ec2de58a-8834-a46f-edb5-8dbd162ed3bb BootID:7592da1f-01ca-4f50-a698-088f17371698 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fe:1c:79:37:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fe:1c:79:37:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:9a:5f:48:e9:23 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 26 15:38:04.863852 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.863848    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 26 15:38:04.863961 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.863934    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 26 15:38:04.866325 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.866300    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 26 15:38:04.866458 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.866329    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-10.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 26 15:38:04.867611 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.867600    2560 topology_manager.go:138] "Creating topology manager with none policy"
Jul 26 15:38:04.867611 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.867612    2560 container_manager_linux.go:309] "Creating device plugin manager"
Jul 26 15:38:04.867720 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.867619    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 15:38:04.867720 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.867635    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 15:38:04.868404 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.868395    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 26 15:38:04.869761 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.869746    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 15:38:04.870974 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.870963    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 15:38:04.871092 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.871082    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 26 15:38:04.871121 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.871109    2560 kubelet.go:486] "Attempting to sync node with API server"
Jul 26 15:38:04.871154 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.871132    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 26 15:38:04.871154 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.871144    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 26 15:38:04.871212 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.871203    2560 kubelet.go:398] "Adding apiserver pod source"
Jul 26 15:38:04.871238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.871212    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 26 15:38:04.872400 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.872380    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 15:38:04.872471 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.872411    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 15:38:04.875330 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.875314    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 26 15:38:04.877296 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.877282    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 26 15:38:04.877384 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.877318    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 26 15:38:04.879147 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879133    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 26 15:38:04.879215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879152    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 26 15:38:04.879215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879161    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 26 15:38:04.879215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879169    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 26 15:38:04.879215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879180    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 26 15:38:04.879215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879188    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 26 15:38:04.879215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879199    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 26 15:38:04.879215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879206    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 26 15:38:04.879215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879215    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 26 15:38:04.879449 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879222    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 26 15:38:04.879449 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879235    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 26 15:38:04.879449 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879243    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 26 15:38:04.879753 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879743    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 26 15:38:04.879799 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.879754    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 26 15:38:04.880467 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:04.880430    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 26 15:38:04.880467 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:04.880424    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 26 15:38:04.881943 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.881927    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 26 15:38:04.882629 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.882619    2560 server.go:1268] "Started kubelet"
Jul 26 15:38:04.882816 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.882717    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 26 15:38:04.882816 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.882716    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 26 15:38:04.882816 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.882786    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 26 15:38:04.883412 ip-10-0-132-10 systemd[1]: Started Kubernetes Kubelet.
Jul 26 15:38:04.884948 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.884931    2560 server.go:310] "Adding debug handlers to kubelet server"
Jul 26 15:38:04.886111 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.885241    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 26 15:38:04.888012 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.887992    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4b2vc"
Jul 26 15:38:04.888136 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:04.888108    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 26 15:38:04.892185 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.892169    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 26 15:38:04.892185 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.892171    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 26 15:38:04.892390 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:04.892369    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:04.892763 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.892750    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 26 15:38:04.893598 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.893575    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 26 15:38:04.893687 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.893539    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 26 15:38:04.893687 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:04.891775    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-10.ec2.internal.18c5e10fb3faf054  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-10.ec2.internal,UID:ip-10-0-132-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-10.ec2.internal,},FirstTimestamp:2026-07-26 15:38:04.882595924 +0000 UTC m=+0.381985752,LastTimestamp:2026-07-26 15:38:04.882595924 +0000 UTC m=+0.381985752,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-10.ec2.internal,}"
Jul 26 15:38:04.893797 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.893780    2560 reconstruct.go:97] "Volume reconstruction finished"
Jul 26 15:38:04.893797 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.893787    2560 reconciler.go:29] "Reconciler: start to sync state"
Jul 26 15:38:04.893883 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.893835    2560 factory.go:55] Registering systemd factory
Jul 26 15:38:04.893883 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.893875    2560 factory.go:223] Registration of the systemd container factory successfully
Jul 26 15:38:04.894474 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.894457    2560 factory.go:153] Registering CRI-O factory
Jul 26 15:38:04.894474 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.894475    2560 factory.go:223] Registration of the crio container factory successfully
Jul 26 15:38:04.894603 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.894523    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 26 15:38:04.894603 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.894545    2560 factory.go:103] Registering Raw factory
Jul 26 15:38:04.894603 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.894559    2560 manager.go:1196] Started watching for new ooms in manager
Jul 26 15:38:04.895156 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.895141    2560 manager.go:349] Starting recovery of all containers
Jul 26 15:38:04.895368 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.895352    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4b2vc"
Jul 26 15:38:04.897436 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:04.897214    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 26 15:38:04.897436 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:04.897333    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 26 15:38:04.909010 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.908994    2560 manager.go:354] Recovery completed
Jul 26 15:38:04.912704 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.912691    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:04.914540 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.914525    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:04.914613 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.914554    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:04.914613 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.914567    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:04.915077 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.915063    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 26 15:38:04.915077 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.915076    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 26 15:38:04.915194 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.915077    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:04.915194 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.915092    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 15:38:04.915726 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.915712    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:04.915768 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.915738    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:04.915768 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.915750    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:04.917746 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.917734    2560 policy_none.go:49] "None policy: Start"
Jul 26 15:38:04.917806 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.917751    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 26 15:38:04.917806 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.917764    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 26 15:38:04.919767 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.919754    2560 policy_none.go:47] "Start"
Jul 26 15:38:04.958848 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.958835    2560 manager.go:337] "Starting Device Plugin manager"
Jul 26 15:38:04.958916 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:04.958870    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 26 15:38:04.958916 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.958882    2560 server.go:87] "Starting device plugin registration server"
Jul 26 15:38:04.959136 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.959124    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 26 15:38:04.959200 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.959158    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 26 15:38:04.959255 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.959243    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 26 15:38:04.959435 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.959321    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 26 15:38:04.959435 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:04.959332    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 26 15:38:04.959786 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:04.959755    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 26 15:38:04.959852 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:04.959812    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.059297 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.059275    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:05.061535 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.061521    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:05.061608 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.061550    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:05.061608 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.061571    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:05.061608 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.061606    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.068033 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.067992    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.068033 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.068012    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-10.ec2.internal\": node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.084561 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.084545    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.131212 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.131184    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 26 15:38:05.133047 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.132304    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 26 15:38:05.133047 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.132334    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 26 15:38:05.133047 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.132349    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 26 15:38:05.133047 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.132376    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 26 15:38:05.134738 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.134722    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:05.184803 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.184785    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.233248 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.233228    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal"]
Jul 26 15:38:05.233308 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.233302    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:05.234068 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.234049    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:05.234116 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.234079    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:05.234116 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.234088    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:05.236110 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.236099    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:05.236246 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.236234    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.236277 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.236259    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:05.236746 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.236731    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:05.236746 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.236730    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:05.236856 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.236757    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:05.236856 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.236762    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:05.236856 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.236774    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:05.236856 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.236775    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:05.238753 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.238741    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.238812 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.238763    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:05.239398 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.239383    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:05.239462 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.239413    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:05.239462 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.239438    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:05.253400 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.253383    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-10.ec2.internal\" not found" node="ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.256612 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.256594    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-10.ec2.internal\" not found" node="ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.284866 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.284850    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.295188 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.295167    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68a25c36792e8d47871b7d498092399a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"68a25c36792e8d47871b7d498092399a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.295268 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.295191    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a25c36792e8d47871b7d498092399a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"68a25c36792e8d47871b7d498092399a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.295268 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.295206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/660fa30c874fcff3f1c5867b1497b385-config\") pod \"kube-apiserver-proxy-ip-10-0-132-10.ec2.internal\" (UID: \"660fa30c874fcff3f1c5867b1497b385\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.385364 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.385321    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.395711 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.395693    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/660fa30c874fcff3f1c5867b1497b385-config\") pod \"kube-apiserver-proxy-ip-10-0-132-10.ec2.internal\" (UID: \"660fa30c874fcff3f1c5867b1497b385\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.395766 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.395714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68a25c36792e8d47871b7d498092399a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"68a25c36792e8d47871b7d498092399a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.395766 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.395730    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a25c36792e8d47871b7d498092399a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"68a25c36792e8d47871b7d498092399a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.395839 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.395774    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/660fa30c874fcff3f1c5867b1497b385-config\") pod \"kube-apiserver-proxy-ip-10-0-132-10.ec2.internal\" (UID: \"660fa30c874fcff3f1c5867b1497b385\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.395839 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.395799    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a25c36792e8d47871b7d498092399a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"68a25c36792e8d47871b7d498092399a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.395839 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.395783    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68a25c36792e8d47871b7d498092399a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal\" (UID: \"68a25c36792e8d47871b7d498092399a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.485870 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.485844    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.556360 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.556344    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.561253 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.561177    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:05.586140 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.586119    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.686703 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.686675    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.787234 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.787215    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.819108 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.819085    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:05.838133 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.838110    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 26 15:38:05.838241 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.838223    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 15:38:05.838292 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.838258    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 15:38:05.887457 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.887440    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:05.890973 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.890957    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:05.901307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.901281    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-25 15:33:04 +0000 UTC" deadline="2028-03-02 08:23:38.95311885 +0000 UTC"
Jul 26 15:38:05.901307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:05.901303    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14032h45m33.05181867s"
Jul 26 15:38:05.987572 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:05.987554    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:06.088190 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:06.088162    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-10.ec2.internal\" not found"
Jul 26 15:38:06.103883 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.103866    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:06.111409 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:06.111380    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a25c36792e8d47871b7d498092399a.slice/crio-a34302f4f33049b95dd65439ec0bfd61c6caf29a75f190f2277ed0e2929efc1f WatchSource:0}: Error finding container a34302f4f33049b95dd65439ec0bfd61c6caf29a75f190f2277ed0e2929efc1f: Status 404 returned error can't find the container with id a34302f4f33049b95dd65439ec0bfd61c6caf29a75f190f2277ed0e2929efc1f
Jul 26 15:38:06.111754 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:06.111732    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod660fa30c874fcff3f1c5867b1497b385.slice/crio-d0fbdf1f35ae18dca0f2c4df84b6eac0dbfc8593bef750b03593a2d5a45b9259 WatchSource:0}: Error finding container d0fbdf1f35ae18dca0f2c4df84b6eac0dbfc8593bef750b03593a2d5a45b9259: Status 404 returned error can't find the container with id d0fbdf1f35ae18dca0f2c4df84b6eac0dbfc8593bef750b03593a2d5a45b9259
Jul 26 15:38:06.117252 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.117238    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:38:06.135444 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.135406    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" event={"ID":"660fa30c874fcff3f1c5867b1497b385","Type":"ContainerStarted","Data":"d0fbdf1f35ae18dca0f2c4df84b6eac0dbfc8593bef750b03593a2d5a45b9259"}
Jul 26 15:38:06.136392 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.136371    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" event={"ID":"68a25c36792e8d47871b7d498092399a","Type":"ContainerStarted","Data":"a34302f4f33049b95dd65439ec0bfd61c6caf29a75f190f2277ed0e2929efc1f"}
Jul 26 15:38:06.192844 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.192824    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:06.203114 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.203077    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 15:38:06.204417 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.204405    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal"
Jul 26 15:38:06.212122 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.212106    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 15:38:06.651366 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.651223    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:06.871390 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.871353    2560 apiserver.go:52] "Watching apiserver"
Jul 26 15:38:06.877573 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.877542    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 26 15:38:06.878108 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.878083    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-4dxwj","kube-system/konnectivity-agent-rx9nc","kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5","openshift-dns/node-resolver-tlcgb","openshift-multus/multus-d7sqd","openshift-network-diagnostics/network-check-target-lkfjb","openshift-network-operator/iptables-alerter-dbdsm","openshift-cluster-node-tuning-operator/tuned-ptzns","openshift-image-registry/node-ca-5cr5p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal","openshift-multus/multus-additional-cni-plugins-mr2zm","openshift-multus/network-metrics-daemon-pglx2","openshift-ovn-kubernetes/ovnkube-node-sd6zj"]
Jul 26 15:38:06.880652 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.880547    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:06.880652 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:06.880633    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:06.882878 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.882834    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:06.882983 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:06.882933    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:06.887181 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.885506    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.888897 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.887861    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 26 15:38:06.888897 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.888080    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-prt8g\""
Jul 26 15:38:06.888897 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.888249    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:06.888897 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.888840    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 26 15:38:06.889185 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.888941    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 26 15:38:06.889185 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.889005    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 26 15:38:06.889819 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.889790    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 26 15:38:06.889926 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.889892    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 26 15:38:06.890494 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.890475    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 26 15:38:06.890725 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.890650    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 26 15:38:06.890810 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.890752    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 26 15:38:06.890871 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.890845    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tfj84\""
Jul 26 15:38:06.891715 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.891698    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:06.892438 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.892418    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 26 15:38:06.893447 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.893426    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 26 15:38:06.893574 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.893480    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kdph2\""
Jul 26 15:38:06.893705 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.893687    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 26 15:38:06.895190 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.895171    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.896786 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.896768    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 26 15:38:06.897440 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.897404    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:06.897781 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.897760    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 26 15:38:06.897865 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.897811    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 26 15:38:06.897999 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.897824    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wp755\""
Jul 26 15:38:06.897999 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.897928    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 26 15:38:06.899403 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.899384    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7k9gr\""
Jul 26 15:38:06.899574 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.899551    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 26 15:38:06.899717 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.899586    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 26 15:38:06.899717 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.899698    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:06.899817 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:06.899762    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:06.902190 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902116    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:06.902397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902368    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3132b60a-04c3-4270-9815-5dd0f87046ec-cni-binary-copy\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.902503 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-var-lib-kubelet\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.902503 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902469    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-daemon-config\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.902503 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902496    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-systemd-units\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.902721 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902522    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-cni-bin\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.902721 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902550    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjdj8\" (UniqueName: \"kubernetes.io/projected/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-kube-api-access-fjdj8\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.902721 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902575    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-sys-fs\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:06.902721 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902604    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-conf-dir\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.902721 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902642    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75lfb\" (UniqueName: \"kubernetes.io/projected/3132b60a-04c3-4270-9815-5dd0f87046ec-kube-api-access-75lfb\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.902721 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902667    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-run-netns\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.902721 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902695    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-run-openvswitch\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.902721 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902719    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-node-log\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902741    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-system-cni-dir\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902799    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-run-multus-certs\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902831    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-etc-kubernetes\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902855    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s9xg\" (UniqueName: \"kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg\") pod \"network-check-target-lkfjb\" (UID: \"596c9959-6728-4c8f-8c65-faec8b068891\") " pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902872    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902882    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.902971    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-slash\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903091    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-cnibin\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903113    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-run-netns\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903151    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-var-lib-cni-multus\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903183    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-run-systemd\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903220 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903210    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-ovnkube-script-lib\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903233    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-registration-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903262    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c04d11-ba9f-4142-906a-1e10f3ad0954-tmp-dir\") pod \"node-resolver-tlcgb\" (UID: \"a8c04d11-ba9f-4142-906a-1e10f3ad0954\") " pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903287    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-var-lib-cni-bin\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903310    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-run-ovn\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903337    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89hvf\" (UniqueName: \"kubernetes.io/projected/a8c04d11-ba9f-4142-906a-1e10f3ad0954-kube-api-access-89hvf\") pod \"node-resolver-tlcgb\" (UID: \"a8c04d11-ba9f-4142-906a-1e10f3ad0954\") " pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903373    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-run-k8s-cni-cncf-io\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903397    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-hostroot\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903419    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-var-lib-openvswitch\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903466    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-log-socket\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903511    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903552    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-cni-netd\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903581    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a8c04d11-ba9f-4142-906a-1e10f3ad0954-hosts-file\") pod \"node-resolver-tlcgb\" (UID: \"a8c04d11-ba9f-4142-906a-1e10f3ad0954\") " pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903606    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-socket-dir-parent\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903651    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-env-overrides\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903700    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-ovn-node-metrics-cert\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.903970 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903725    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-socket-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903780    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-device-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903807    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903858    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-cni-dir\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903881    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-os-release\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903942    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhsnk\" (UniqueName: \"kubernetes.io/projected/89226f34-999b-4a77-a5f7-44f8c026ef55-kube-api-access-rhsnk\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.903989    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-kubelet\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.904018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-etc-openvswitch\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.904042    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-ovnkube-config\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.904065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.904089    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4456m\" (UniqueName: \"kubernetes.io/projected/5ec8eed3-217d-4636-bb62-f5b48aa645b5-kube-api-access-4456m\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.904273    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.904466    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-28cnb\""
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.904502    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 26 15:38:06.904891 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.904580    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:06.906399 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.906379    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 26 15:38:06.906562 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.906541    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 26 15:38:06.906638 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.906615    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w6m7w\""
Jul 26 15:38:06.906939 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.906923    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 26 15:38:06.907119 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.907023    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:06.908715 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.908687    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 26 15:38:06.909043 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.909022    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 26 15:38:06.909135 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.909105    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cksct\""
Jul 26 15:38:06.909237 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.909202    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 26 15:38:06.909563 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.909546    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:06.911742 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.911719    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tf5pw\""
Jul 26 15:38:06.912077 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.912056    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 26 15:38:06.912174 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.912091    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 26 15:38:06.921234 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.921094    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pdfnc"
Jul 26 15:38:06.932734 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.932377    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pdfnc"
Jul 26 15:38:06.994971 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:06.994947    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 26 15:38:07.004538 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004378    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-os-release\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.004538 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004415    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhsnk\" (UniqueName: \"kubernetes.io/projected/89226f34-999b-4a77-a5f7-44f8c026ef55-kube-api-access-rhsnk\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:07.004538 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004443    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-ovnkube-config\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.004538 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.004538 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004486    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4456m\" (UniqueName: \"kubernetes.io/projected/5ec8eed3-217d-4636-bb62-f5b48aa645b5-kube-api-access-4456m\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.004538 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004509    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3132b60a-04c3-4270-9815-5dd0f87046ec-cni-binary-copy\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.004538 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004531    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-var-lib-kubelet\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004553    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-daemon-config\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004578    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-systemd-units\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-cni-bin\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004650    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-cni-bin\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004652    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-systemd-units\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-var-lib-kubelet\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004652    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-os-release\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004718    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004761    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjdj8\" (UniqueName: \"kubernetes.io/projected/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-kube-api-access-fjdj8\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004782    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-sys-fs\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004803    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-conf-dir\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004826    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-node-log\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004858    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff85593b-fd58-4b0b-bfd6-8b378ca48af1-konnectivity-ca\") pod \"konnectivity-agent-rx9nc\" (UID: \"ff85593b-fd58-4b0b-bfd6-8b378ca48af1\") " pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004877    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-sys-fs\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-sysctl-conf\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004939    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-node-log\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004949    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-systemd\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.005008 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004976    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-conf-dir\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.004977    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-tuned\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005008    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ead3ac8-eea8-41eb-b1a4-7198411db6bc-iptables-alerter-script\") pod \"iptables-alerter-dbdsm\" (UID: \"8ead3ac8-eea8-41eb-b1a4-7198411db6bc\") " pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005030    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc99l\" (UniqueName: \"kubernetes.io/projected/8ead3ac8-eea8-41eb-b1a4-7198411db6bc-kube-api-access-wc99l\") pod \"iptables-alerter-dbdsm\" (UID: \"8ead3ac8-eea8-41eb-b1a4-7198411db6bc\") " pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-run-multus-certs\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005077    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005139    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-run-multus-certs\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005181    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-modprobe-d\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.005193    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005211    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-kubernetes\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005239    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-lib-modules\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005270    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-cnibin\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005268    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-ovnkube-config\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005310    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6xnx\" (UniqueName: \"kubernetes.io/projected/e15fb8a5-0274-49d5-a081-6b7097b526e1-kube-api-access-f6xnx\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.005329    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs podName:89226f34-999b-4a77-a5f7-44f8c026ef55 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:07.505312715 +0000 UTC m=+3.004702531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs") pod "network-metrics-daemon-pglx2" (UID: "89226f34-999b-4a77-a5f7-44f8c026ef55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005344    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3132b60a-04c3-4270-9815-5dd0f87046ec-cni-binary-copy\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005367    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-run-netns\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.005876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005377    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-daemon-config\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-run-systemd\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-ovnkube-script-lib\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005451    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-run-systemd\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005453    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-registration-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005467    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-run-netns\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005498    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-registration-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005497    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a53b35cf-9867-4dd7-9e53-1cbab673f3fe-serviceca\") pod \"node-ca-5cr5p\" (UID: \"a53b35cf-9867-4dd7-9e53-1cbab673f3fe\") " pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005535    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-host\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005559    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e15fb8a5-0274-49d5-a081-6b7097b526e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005583    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c04d11-ba9f-4142-906a-1e10f3ad0954-tmp-dir\") pod \"node-resolver-tlcgb\" (UID: \"a8c04d11-ba9f-4142-906a-1e10f3ad0954\") " pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005607    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-var-lib-cni-bin\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-run-ovn\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005650    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/357d29fb-e72d-43f6-88c7-6c014abbd9c8-tmp\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005680    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-run-ovn\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005680    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-var-lib-cni-bin\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.006507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005702    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89hvf\" (UniqueName: \"kubernetes.io/projected/a8c04d11-ba9f-4142-906a-1e10f3ad0954-kube-api-access-89hvf\") pod \"node-resolver-tlcgb\" (UID: \"a8c04d11-ba9f-4142-906a-1e10f3ad0954\") " pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005732    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-socket-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005764    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005796    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wjwr\" (UniqueName: \"kubernetes.io/projected/357d29fb-e72d-43f6-88c7-6c014abbd9c8-kube-api-access-9wjwr\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005919    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e15fb8a5-0274-49d5-a081-6b7097b526e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005934    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-etc-selinux\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005968    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-cni-dir\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005971    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c04d11-ba9f-4142-906a-1e10f3ad0954-tmp-dir\") pod \"node-resolver-tlcgb\" (UID: \"a8c04d11-ba9f-4142-906a-1e10f3ad0954\") " pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.005924    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-socket-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006042    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-ovnkube-script-lib\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006055    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-kubelet\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006010    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-kubelet\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006094    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-etc-openvswitch\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a53b35cf-9867-4dd7-9e53-1cbab673f3fe-host\") pod \"node-ca-5cr5p\" (UID: \"a53b35cf-9867-4dd7-9e53-1cbab673f3fe\") " pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006143    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-cni-dir\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-run\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006177    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-etc-openvswitch\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006203    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-sys\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006273    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-dbus\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006315    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-system-cni-dir\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006344    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e15fb8a5-0274-49d5-a081-6b7097b526e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75lfb\" (UniqueName: \"kubernetes.io/projected/3132b60a-04c3-4270-9815-5dd0f87046ec-kube-api-access-75lfb\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006397    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-run-netns\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006420    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-run-openvswitch\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006450    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-run-netns\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006460    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff85593b-fd58-4b0b-bfd6-8b378ca48af1-agent-certs\") pod \"konnectivity-agent-rx9nc\" (UID: \"ff85593b-fd58-4b0b-bfd6-8b378ca48af1\") " pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006499    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-system-cni-dir\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-etc-kubernetes\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006559    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-run-openvswitch\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006560    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s9xg\" (UniqueName: \"kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg\") pod \"network-check-target-lkfjb\" (UID: \"596c9959-6728-4c8f-8c65-faec8b068891\") " pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006567    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-system-cni-dir\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006573    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-etc-kubernetes\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006604    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-slash\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006631    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.007886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006652    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-slash\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006661    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006685    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-cnibin\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006694    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-var-lib-cni-multus\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006737    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-sysctl-d\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006754    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-var-lib-cni-multus\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006762    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-cnibin\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-sysconfig\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006809    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-kubelet-config\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006839    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-run-k8s-cni-cncf-io\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006865    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-hostroot\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006903    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-var-lib-openvswitch\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006918    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-host-run-k8s-cni-cncf-io\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006928    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-log-socket\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006945    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-var-lib-openvswitch\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006943    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-hostroot\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.008397 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006977    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-log-socket\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006978    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-cni-netd\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.006999    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-run-ovn-kubernetes\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007013    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-host-cni-netd\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007020    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-ovn-node-metrics-cert\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-device-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007070    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a8c04d11-ba9f-4142-906a-1e10f3ad0954-hosts-file\") pod \"node-resolver-tlcgb\" (UID: \"a8c04d11-ba9f-4142-906a-1e10f3ad0954\") " pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007095    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-socket-dir-parent\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007111    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ec8eed3-217d-4636-bb62-f5b48aa645b5-device-dir\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007119    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-env-overrides\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007137    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a8c04d11-ba9f-4142-906a-1e10f3ad0954-hosts-file\") pod \"node-resolver-tlcgb\" (UID: \"a8c04d11-ba9f-4142-906a-1e10f3ad0954\") " pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs8p7\" (UniqueName: \"kubernetes.io/projected/a53b35cf-9867-4dd7-9e53-1cbab673f3fe-kube-api-access-xs8p7\") pod \"node-ca-5cr5p\" (UID: \"a53b35cf-9867-4dd7-9e53-1cbab673f3fe\") " pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007152    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3132b60a-04c3-4270-9815-5dd0f87046ec-multus-socket-dir-parent\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007235    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-var-lib-kubelet\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ead3ac8-eea8-41eb-b1a4-7198411db6bc-host-slash\") pod \"iptables-alerter-dbdsm\" (UID: \"8ead3ac8-eea8-41eb-b1a4-7198411db6bc\") " pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007268    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-os-release\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.008931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007403    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 26 15:38:07.009416 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.007542    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-env-overrides\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.010482 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.010464    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-ovn-node-metrics-cert\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.013301 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.013280    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:07.013399 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.013305    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:07.013399 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.013317    2560 projected.go:196] Error preparing data for projected volume kube-api-access-7s9xg for pod openshift-network-diagnostics/network-check-target-lkfjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:07.013399 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.013398    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg podName:596c9959-6728-4c8f-8c65-faec8b068891 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:07.513383544 +0000 UTC m=+3.012773361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7s9xg" (UniqueName: "kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg") pod "network-check-target-lkfjb" (UID: "596c9959-6728-4c8f-8c65-faec8b068891") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:07.015035 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.015006    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4456m\" (UniqueName: \"kubernetes.io/projected/5ec8eed3-217d-4636-bb62-f5b48aa645b5-kube-api-access-4456m\") pod \"aws-ebs-csi-driver-node-dwbx5\" (UID: \"5ec8eed3-217d-4636-bb62-f5b48aa645b5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.015832 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.015786    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75lfb\" (UniqueName: \"kubernetes.io/projected/3132b60a-04c3-4270-9815-5dd0f87046ec-kube-api-access-75lfb\") pod \"multus-d7sqd\" (UID: \"3132b60a-04c3-4270-9815-5dd0f87046ec\") " pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.015971 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.015950    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhsnk\" (UniqueName: \"kubernetes.io/projected/89226f34-999b-4a77-a5f7-44f8c026ef55-kube-api-access-rhsnk\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:07.016605 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.016534    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjdj8\" (UniqueName: \"kubernetes.io/projected/2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6-kube-api-access-fjdj8\") pod \"ovnkube-node-sd6zj\" (UID: \"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6\") " pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.016690 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.016645    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89hvf\" (UniqueName: \"kubernetes.io/projected/a8c04d11-ba9f-4142-906a-1e10f3ad0954-kube-api-access-89hvf\") pod \"node-resolver-tlcgb\" (UID: \"a8c04d11-ba9f-4142-906a-1e10f3ad0954\") " pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:07.108031 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108008    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a53b35cf-9867-4dd7-9e53-1cbab673f3fe-host\") pod \"node-ca-5cr5p\" (UID: \"a53b35cf-9867-4dd7-9e53-1cbab673f3fe\") " pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:07.108031 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108034    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-run\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.108206 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108050    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-sys\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.108206 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108066    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-dbus\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:07.108206 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108125    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-run\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.108206 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108147    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a53b35cf-9867-4dd7-9e53-1cbab673f3fe-host\") pod \"node-ca-5cr5p\" (UID: \"a53b35cf-9867-4dd7-9e53-1cbab673f3fe\") " pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:07.108206 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108158    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-system-cni-dir\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.108206 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108158    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-sys\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.108206 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108192    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-dbus\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:07.108206 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108193    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-system-cni-dir\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.108206 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108199    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e15fb8a5-0274-49d5-a081-6b7097b526e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108230    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff85593b-fd58-4b0b-bfd6-8b378ca48af1-agent-certs\") pod \"konnectivity-agent-rx9nc\" (UID: \"ff85593b-fd58-4b0b-bfd6-8b378ca48af1\") " pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108260    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108285    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-sysctl-d\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.108391    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108398    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-sysctl-d\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108403    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-sysconfig\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108440    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-kubelet-config\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108446    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-sysconfig\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.108456    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret podName:9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f nodeName:}" failed. No retries permitted until 2026-07-26 15:38:07.608438593 +0000 UTC m=+3.107828413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret") pod "global-pull-secret-syncer-4dxwj" (UID: "9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108494    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-kubelet-config\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108503    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs8p7\" (UniqueName: \"kubernetes.io/projected/a53b35cf-9867-4dd7-9e53-1cbab673f3fe-kube-api-access-xs8p7\") pod \"node-ca-5cr5p\" (UID: \"a53b35cf-9867-4dd7-9e53-1cbab673f3fe\") " pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108532    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-var-lib-kubelet\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108557    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ead3ac8-eea8-41eb-b1a4-7198411db6bc-host-slash\") pod \"iptables-alerter-dbdsm\" (UID: \"8ead3ac8-eea8-41eb-b1a4-7198411db6bc\") " pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:07.108607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108580    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-os-release\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108604    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-var-lib-kubelet\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff85593b-fd58-4b0b-bfd6-8b378ca48af1-konnectivity-ca\") pod \"konnectivity-agent-rx9nc\" (UID: \"ff85593b-fd58-4b0b-bfd6-8b378ca48af1\") " pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108650    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-sysctl-conf\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108656    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ead3ac8-eea8-41eb-b1a4-7198411db6bc-host-slash\") pod \"iptables-alerter-dbdsm\" (UID: \"8ead3ac8-eea8-41eb-b1a4-7198411db6bc\") " pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108673    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-systemd\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108694    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-os-release\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108716    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-tuned\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108718    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e15fb8a5-0274-49d5-a081-6b7097b526e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108741    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-systemd\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108743    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ead3ac8-eea8-41eb-b1a4-7198411db6bc-iptables-alerter-script\") pod \"iptables-alerter-dbdsm\" (UID: \"8ead3ac8-eea8-41eb-b1a4-7198411db6bc\") " pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108782    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc99l\" (UniqueName: \"kubernetes.io/projected/8ead3ac8-eea8-41eb-b1a4-7198411db6bc-kube-api-access-wc99l\") pod \"iptables-alerter-dbdsm\" (UID: \"8ead3ac8-eea8-41eb-b1a4-7198411db6bc\") " pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-sysctl-conf\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108831    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-modprobe-d\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108854    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-kubernetes\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108878    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-lib-modules\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108923    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-cnibin\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.109307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108950    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6xnx\" (UniqueName: \"kubernetes.io/projected/e15fb8a5-0274-49d5-a081-6b7097b526e1-kube-api-access-f6xnx\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108978    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a53b35cf-9867-4dd7-9e53-1cbab673f3fe-serviceca\") pod \"node-ca-5cr5p\" (UID: \"a53b35cf-9867-4dd7-9e53-1cbab673f3fe\") " pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109002    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-host\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109016    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-cnibin\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109023    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e15fb8a5-0274-49d5-a081-6b7097b526e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109059    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/357d29fb-e72d-43f6-88c7-6c014abbd9c8-tmp\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109074    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-kubernetes\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109089    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109113    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff85593b-fd58-4b0b-bfd6-8b378ca48af1-konnectivity-ca\") pod \"konnectivity-agent-rx9nc\" (UID: \"ff85593b-fd58-4b0b-bfd6-8b378ca48af1\") " pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109123    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wjwr\" (UniqueName: \"kubernetes.io/projected/357d29fb-e72d-43f6-88c7-6c014abbd9c8-kube-api-access-9wjwr\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109157    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e15fb8a5-0274-49d5-a081-6b7097b526e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109193    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-lib-modules\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109348    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e15fb8a5-0274-49d5-a081-6b7097b526e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109371    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e15fb8a5-0274-49d5-a081-6b7097b526e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.108978    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-modprobe-d\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109438    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/357d29fb-e72d-43f6-88c7-6c014abbd9c8-host\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109604    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e15fb8a5-0274-49d5-a081-6b7097b526e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.110086 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109806    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a53b35cf-9867-4dd7-9e53-1cbab673f3fe-serviceca\") pod \"node-ca-5cr5p\" (UID: \"a53b35cf-9867-4dd7-9e53-1cbab673f3fe\") " pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:07.110884 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.109916    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ead3ac8-eea8-41eb-b1a4-7198411db6bc-iptables-alerter-script\") pod \"iptables-alerter-dbdsm\" (UID: \"8ead3ac8-eea8-41eb-b1a4-7198411db6bc\") " pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:07.111313 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.111285    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/357d29fb-e72d-43f6-88c7-6c014abbd9c8-tmp\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.111424 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.111405    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/357d29fb-e72d-43f6-88c7-6c014abbd9c8-etc-tuned\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.111469 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.111457    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff85593b-fd58-4b0b-bfd6-8b378ca48af1-agent-certs\") pod \"konnectivity-agent-rx9nc\" (UID: \"ff85593b-fd58-4b0b-bfd6-8b378ca48af1\") " pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:07.115928 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.115885    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc99l\" (UniqueName: \"kubernetes.io/projected/8ead3ac8-eea8-41eb-b1a4-7198411db6bc-kube-api-access-wc99l\") pod \"iptables-alerter-dbdsm\" (UID: \"8ead3ac8-eea8-41eb-b1a4-7198411db6bc\") " pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:07.116052 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.116031    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs8p7\" (UniqueName: \"kubernetes.io/projected/a53b35cf-9867-4dd7-9e53-1cbab673f3fe-kube-api-access-xs8p7\") pod \"node-ca-5cr5p\" (UID: \"a53b35cf-9867-4dd7-9e53-1cbab673f3fe\") " pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:07.116631 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.116605    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wjwr\" (UniqueName: \"kubernetes.io/projected/357d29fb-e72d-43f6-88c7-6c014abbd9c8-kube-api-access-9wjwr\") pod \"tuned-ptzns\" (UID: \"357d29fb-e72d-43f6-88c7-6c014abbd9c8\") " pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.116724 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.116711    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6xnx\" (UniqueName: \"kubernetes.io/projected/e15fb8a5-0274-49d5-a081-6b7097b526e1-kube-api-access-f6xnx\") pod \"multus-additional-cni-plugins-mr2zm\" (UID: \"e15fb8a5-0274-49d5-a081-6b7097b526e1\") " pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.201243 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.201195    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:07.208953 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.208917    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5"
Jul 26 15:38:07.218424 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.218405    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tlcgb"
Jul 26 15:38:07.225971 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.225948    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mr2zm"
Jul 26 15:38:07.236567 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.236550    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d7sqd"
Jul 26 15:38:07.244209 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.244188    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ptzns"
Jul 26 15:38:07.258768 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.258751    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:07.265303 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.265285    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5cr5p"
Jul 26 15:38:07.271951 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.271927    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dbdsm"
Jul 26 15:38:07.512474 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.512402    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:07.512622 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.512575    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:07.512692 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.512644    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs podName:89226f34-999b-4a77-a5f7-44f8c026ef55 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:08.512627672 +0000 UTC m=+4.012017491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs") pod "network-metrics-daemon-pglx2" (UID: "89226f34-999b-4a77-a5f7-44f8c026ef55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:07.613298 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.613273    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s9xg\" (UniqueName: \"kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg\") pod \"network-check-target-lkfjb\" (UID: \"596c9959-6728-4c8f-8c65-faec8b068891\") " pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:07.613298 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.613303    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:07.613467 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.613398    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:07.613467 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.613420    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:07.613467 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.613440    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:07.613467 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.613450    2560 projected.go:196] Error preparing data for projected volume kube-api-access-7s9xg for pod openshift-network-diagnostics/network-check-target-lkfjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:07.613590 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.613447    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret podName:9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f nodeName:}" failed. No retries permitted until 2026-07-26 15:38:08.613430047 +0000 UTC m=+4.112819867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret") pod "global-pull-secret-syncer-4dxwj" (UID: "9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:07.613590 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:07.613492    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg podName:596c9959-6728-4c8f-8c65-faec8b068891 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:08.613480177 +0000 UTC m=+4.112869993 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7s9xg" (UniqueName: "kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg") pod "network-check-target-lkfjb" (UID: "596c9959-6728-4c8f-8c65-faec8b068891") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:07.661717 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.661588    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:07.936048 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.936017    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 15:33:06 +0000 UTC" deadline="2028-03-25 22:44:46.88824356 +0000 UTC"
Jul 26 15:38:07.936048 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:07.936043    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14599h6m38.95220343s"
Jul 26 15:38:07.959101 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:07.959044    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c04d11_ba9f_4142_906a_1e10f3ad0954.slice/crio-1871f7c9326fc02c7711932b925a91ff85ab4360c6468bfb739250cc2fcd2ae5 WatchSource:0}: Error finding container 1871f7c9326fc02c7711932b925a91ff85ab4360c6468bfb739250cc2fcd2ae5: Status 404 returned error can't find the container with id 1871f7c9326fc02c7711932b925a91ff85ab4360c6468bfb739250cc2fcd2ae5
Jul 26 15:38:07.963767 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:07.963599    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ead3ac8_eea8_41eb_b1a4_7198411db6bc.slice/crio-fe061111ca689f62e71b29b3f6293daeb4c01da1c4db00655c1b1db2649207d3 WatchSource:0}: Error finding container fe061111ca689f62e71b29b3f6293daeb4c01da1c4db00655c1b1db2649207d3: Status 404 returned error can't find the container with id fe061111ca689f62e71b29b3f6293daeb4c01da1c4db00655c1b1db2649207d3
Jul 26 15:38:07.966569 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:07.966543    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15fb8a5_0274_49d5_a081_6b7097b526e1.slice/crio-9325324f01b20b9b3900e637bc663b3d4b000d5f022640e95b48f05b1bea5763 WatchSource:0}: Error finding container 9325324f01b20b9b3900e637bc663b3d4b000d5f022640e95b48f05b1bea5763: Status 404 returned error can't find the container with id 9325324f01b20b9b3900e637bc663b3d4b000d5f022640e95b48f05b1bea5763
Jul 26 15:38:07.967386 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:07.967362    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2985b5a1_4ad3_451a_b8a5_1c2f65bc12a6.slice/crio-aae59f613d75f21869f4f9cdfa498f80e0aa406edcf732123963d947b937069b WatchSource:0}: Error finding container aae59f613d75f21869f4f9cdfa498f80e0aa406edcf732123963d947b937069b: Status 404 returned error can't find the container with id aae59f613d75f21869f4f9cdfa498f80e0aa406edcf732123963d947b937069b
Jul 26 15:38:07.971237 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:07.970962    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff85593b_fd58_4b0b_bfd6_8b378ca48af1.slice/crio-b7135be33b51fbf903aff84f27fd1cd024f44cd75ce0761d3724c8ad42055e26 WatchSource:0}: Error finding container b7135be33b51fbf903aff84f27fd1cd024f44cd75ce0761d3724c8ad42055e26: Status 404 returned error can't find the container with id b7135be33b51fbf903aff84f27fd1cd024f44cd75ce0761d3724c8ad42055e26
Jul 26 15:38:07.974333 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:07.974286    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3132b60a_04c3_4270_9815_5dd0f87046ec.slice/crio-379cdfedd4db4f325c3dc8ffc0f4c2037abc59d9ec64a816e6836fa6ba98b598 WatchSource:0}: Error finding container 379cdfedd4db4f325c3dc8ffc0f4c2037abc59d9ec64a816e6836fa6ba98b598: Status 404 returned error can't find the container with id 379cdfedd4db4f325c3dc8ffc0f4c2037abc59d9ec64a816e6836fa6ba98b598
Jul 26 15:38:08.132630 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.132506    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:08.132735 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.132537    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:08.132735 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.132706    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:08.132874 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.132802    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:08.132874 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.132541    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:08.132998 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.132935    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:08.145231 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.142086    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" event={"ID":"660fa30c874fcff3f1c5867b1497b385","Type":"ContainerStarted","Data":"506692e31867c8d22ff7f0aea3721656c85a7a44a70b6b8ba6b3b747e0d01f6e"}
Jul 26 15:38:08.147232 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.147208    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj" event={"ID":"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6","Type":"ContainerStarted","Data":"aae59f613d75f21869f4f9cdfa498f80e0aa406edcf732123963d947b937069b"}
Jul 26 15:38:08.148554 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.148532    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d7sqd" event={"ID":"3132b60a-04c3-4270-9815-5dd0f87046ec","Type":"ContainerStarted","Data":"379cdfedd4db4f325c3dc8ffc0f4c2037abc59d9ec64a816e6836fa6ba98b598"}
Jul 26 15:38:08.149639 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.149615    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5cr5p" event={"ID":"a53b35cf-9867-4dd7-9e53-1cbab673f3fe","Type":"ContainerStarted","Data":"9cec7c8a3c3e6f8febd5b11db363ec309ec402a8c9edf300143413e6af6b7806"}
Jul 26 15:38:08.150665 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.150640    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ptzns" event={"ID":"357d29fb-e72d-43f6-88c7-6c014abbd9c8","Type":"ContainerStarted","Data":"2ec3d18c048eb686bc91d2519fd4d8d899778b47422b00f163b2d77b636bec1a"}
Jul 26 15:38:08.152526 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.152498    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rx9nc" event={"ID":"ff85593b-fd58-4b0b-bfd6-8b378ca48af1","Type":"ContainerStarted","Data":"b7135be33b51fbf903aff84f27fd1cd024f44cd75ce0761d3724c8ad42055e26"}
Jul 26 15:38:08.154298 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.154266    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5" event={"ID":"5ec8eed3-217d-4636-bb62-f5b48aa645b5","Type":"ContainerStarted","Data":"f34610ffb5891108922fb3e14e6d5529a441f6b7dd7cb8ad49429cd96816d22f"}
Jul 26 15:38:08.154400 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.154345    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-10.ec2.internal" podStartSLOduration=2.154333925 podStartE2EDuration="2.154333925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:08.154084332 +0000 UTC m=+3.653474170" watchObservedRunningTime="2026-07-26 15:38:08.154333925 +0000 UTC m=+3.653723762"
Jul 26 15:38:08.155364 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.155346    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dbdsm" event={"ID":"8ead3ac8-eea8-41eb-b1a4-7198411db6bc","Type":"ContainerStarted","Data":"fe061111ca689f62e71b29b3f6293daeb4c01da1c4db00655c1b1db2649207d3"}
Jul 26 15:38:08.156784 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.156765    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mr2zm" event={"ID":"e15fb8a5-0274-49d5-a081-6b7097b526e1","Type":"ContainerStarted","Data":"9325324f01b20b9b3900e637bc663b3d4b000d5f022640e95b48f05b1bea5763"}
Jul 26 15:38:08.157785 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.157761    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tlcgb" event={"ID":"a8c04d11-ba9f-4142-906a-1e10f3ad0954","Type":"ContainerStarted","Data":"1871f7c9326fc02c7711932b925a91ff85ab4360c6468bfb739250cc2fcd2ae5"}
Jul 26 15:38:08.520861 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.520781    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:08.521043 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.520995    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:08.521108 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.521063    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs podName:89226f34-999b-4a77-a5f7-44f8c026ef55 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:10.521042152 +0000 UTC m=+6.020431969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs") pod "network-metrics-daemon-pglx2" (UID: "89226f34-999b-4a77-a5f7-44f8c026ef55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:08.621584 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.621550    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s9xg\" (UniqueName: \"kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg\") pod \"network-check-target-lkfjb\" (UID: \"596c9959-6728-4c8f-8c65-faec8b068891\") " pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:08.621759 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.621597    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:08.621759 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.621720    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:08.621870 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.621776    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret podName:9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f nodeName:}" failed. No retries permitted until 2026-07-26 15:38:10.621759658 +0000 UTC m=+6.121149481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret") pod "global-pull-secret-syncer-4dxwj" (UID: "9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:08.622195 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.622178    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:08.622270 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.622204    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:08.622270 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.622217    2560 projected.go:196] Error preparing data for projected volume kube-api-access-7s9xg for pod openshift-network-diagnostics/network-check-target-lkfjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:08.622270 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:08.622260    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg podName:596c9959-6728-4c8f-8c65-faec8b068891 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:10.62224613 +0000 UTC m=+6.121635949 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7s9xg" (UniqueName: "kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg") pod "network-check-target-lkfjb" (UID: "596c9959-6728-4c8f-8c65-faec8b068891") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:08.936862 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.936801    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 15:33:06 +0000 UTC" deadline="2028-03-23 09:13:22.505298731 +0000 UTC"
Jul 26 15:38:08.936862 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:08.936857    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14537h35m13.568445045s"
Jul 26 15:38:09.173043 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:09.172361    2560 generic.go:356] "Generic (PLEG): container finished" podID="68a25c36792e8d47871b7d498092399a" containerID="05acfcf4ee03f91837ec3d98d522268ed04987379846fa40ea992d560c8cc752" exitCode=0
Jul 26 15:38:09.173043 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:09.172956    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" event={"ID":"68a25c36792e8d47871b7d498092399a","Type":"ContainerDied","Data":"05acfcf4ee03f91837ec3d98d522268ed04987379846fa40ea992d560c8cc752"}
Jul 26 15:38:10.134454 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:10.133642    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:10.134454 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.133778    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:10.134454 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:10.134172    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:10.134454 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.134267    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:10.134454 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:10.134324    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:10.134454 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.134388    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:10.197417 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:10.196641    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" event={"ID":"68a25c36792e8d47871b7d498092399a","Type":"ContainerStarted","Data":"5ac0a48517225f2e953090ef9f64e7090a3b3808eb87c80167d29e8475894a88"}
Jul 26 15:38:10.541756 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:10.541200    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:10.541756 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.541378    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:10.541756 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.541440    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs podName:89226f34-999b-4a77-a5f7-44f8c026ef55 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:14.541420808 +0000 UTC m=+10.040810626 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs") pod "network-metrics-daemon-pglx2" (UID: "89226f34-999b-4a77-a5f7-44f8c026ef55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:10.644562 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:10.644530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s9xg\" (UniqueName: \"kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg\") pod \"network-check-target-lkfjb\" (UID: \"596c9959-6728-4c8f-8c65-faec8b068891\") " pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:10.644712 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:10.644576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:10.644771 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.644721    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:10.644818 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.644776    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret podName:9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f nodeName:}" failed. No retries permitted until 2026-07-26 15:38:14.644757664 +0000 UTC m=+10.144147481 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret") pod "global-pull-secret-syncer-4dxwj" (UID: "9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:10.644878 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.644835    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:10.644878 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.644851    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:10.644878 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.644863    2560 projected.go:196] Error preparing data for projected volume kube-api-access-7s9xg for pod openshift-network-diagnostics/network-check-target-lkfjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:10.645046 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:10.644926    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg podName:596c9959-6728-4c8f-8c65-faec8b068891 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:14.644913918 +0000 UTC m=+10.144303753 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7s9xg" (UniqueName: "kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg") pod "network-check-target-lkfjb" (UID: "596c9959-6728-4c8f-8c65-faec8b068891") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:12.133260 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:12.133227    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:12.133706 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:12.133352    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:12.133776 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:12.133747    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:12.133867 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:12.133844    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:12.133954 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:12.133924    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:12.134009 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:12.133995    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:14.132818 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:14.132790    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:14.133243 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:14.132799    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:14.133243 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.132925    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:14.133243 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.132965    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:14.133243 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:14.132808    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:14.133243 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.133110    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:14.580579 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:14.580550    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:14.580759 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.580708    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:14.580821 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.580763    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs podName:89226f34-999b-4a77-a5f7-44f8c026ef55 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:22.580747558 +0000 UTC m=+18.080137373 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs") pod "network-metrics-daemon-pglx2" (UID: "89226f34-999b-4a77-a5f7-44f8c026ef55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:14.681579 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:14.681542    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s9xg\" (UniqueName: \"kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg\") pod \"network-check-target-lkfjb\" (UID: \"596c9959-6728-4c8f-8c65-faec8b068891\") " pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:14.681706 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:14.681588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:14.681752 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.681705    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:14.681752 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.681726    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:14.681752 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.681739    2560 projected.go:196] Error preparing data for projected volume kube-api-access-7s9xg for pod openshift-network-diagnostics/network-check-target-lkfjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:14.681858 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.681746    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:14.681858 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.681792    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg podName:596c9959-6728-4c8f-8c65-faec8b068891 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:22.681772984 +0000 UTC m=+18.181162802 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7s9xg" (UniqueName: "kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg") pod "network-check-target-lkfjb" (UID: "596c9959-6728-4c8f-8c65-faec8b068891") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:14.681858 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:14.681813    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret podName:9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f nodeName:}" failed. No retries permitted until 2026-07-26 15:38:22.681802813 +0000 UTC m=+18.181192630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret") pod "global-pull-secret-syncer-4dxwj" (UID: "9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:16.133010 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:16.132929    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:16.133010 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:16.132962    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:16.133479 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:16.132929    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:16.133479 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:16.133061    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:16.133479 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:16.133162    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:16.133479 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:16.133241    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:18.133153 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:18.133121    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:18.133153 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:18.133146    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:18.133518 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:18.133124    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:18.133518 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:18.133218    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:18.133518 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:18.133287    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:18.133518 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:18.133346    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:20.132866 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:20.132837    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:20.132866 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:20.132866    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:20.133363 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:20.132841    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:20.133363 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:20.132974    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:20.133363 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:20.133107    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:20.133363 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:20.133216    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:22.132832 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:22.132803    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:22.133301 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:22.132948    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:22.133301 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.132961    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:22.133301 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.133031    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:22.133301 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:22.133069    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:22.133301 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.133116    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:22.646946 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:22.646857    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:22.647082 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.646999    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:22.647082 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.647067    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs podName:89226f34-999b-4a77-a5f7-44f8c026ef55 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:38.64704662 +0000 UTC m=+34.146436447 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs") pod "network-metrics-daemon-pglx2" (UID: "89226f34-999b-4a77-a5f7-44f8c026ef55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:22.747908 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:22.747864    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s9xg\" (UniqueName: \"kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg\") pod \"network-check-target-lkfjb\" (UID: \"596c9959-6728-4c8f-8c65-faec8b068891\") " pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:22.748032 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:22.747919    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:22.748032 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.748017    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:22.748150 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.748040    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:22.748150 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.748056    2560 projected.go:196] Error preparing data for projected volume kube-api-access-7s9xg for pod openshift-network-diagnostics/network-check-target-lkfjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:22.748150 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.748104    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:22.748150 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.748118    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg podName:596c9959-6728-4c8f-8c65-faec8b068891 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:38.748102543 +0000 UTC m=+34.247492380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7s9xg" (UniqueName: "kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg") pod "network-check-target-lkfjb" (UID: "596c9959-6728-4c8f-8c65-faec8b068891") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:22.748303 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:22.748162    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret podName:9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f nodeName:}" failed. No retries permitted until 2026-07-26 15:38:38.748144919 +0000 UTC m=+34.247534756 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret") pod "global-pull-secret-syncer-4dxwj" (UID: "9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:24.133188 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:24.133156    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:24.133575 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:24.133156    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:24.133575 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:24.133273    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:24.133575 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:24.133377    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:24.133575 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:24.133156    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:24.133575 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:24.133449    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:26.133779 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.133599    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:26.134210 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:26.133879    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:26.134210 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.133750    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:26.134210 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.133706    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:26.134210 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:26.133992    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:26.134210 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:26.134100    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:26.228652 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.227580    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj" event={"ID":"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6","Type":"ContainerStarted","Data":"6d43d8bea5fe726c30d7f6cdbb7b89f2e56a3b1a27898f527f6b521f065b5ea9"}
Jul 26 15:38:26.228652 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.227647    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj" event={"ID":"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6","Type":"ContainerStarted","Data":"93bb8346232c181bbc7c979af140b11e877b6284ca3852d2fbf8c944168b49a8"}
Jul 26 15:38:26.229793 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.229753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d7sqd" event={"ID":"3132b60a-04c3-4270-9815-5dd0f87046ec","Type":"ContainerStarted","Data":"11ff0e7e53e1d894649443b7237831cde153b631040a3317dcf2c9e9dd5685c6"}
Jul 26 15:38:26.231433 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.231353    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5cr5p" event={"ID":"a53b35cf-9867-4dd7-9e53-1cbab673f3fe","Type":"ContainerStarted","Data":"ece3e20ae01b746e309d9f94b4d1286f5febf62c2c80223ea08ae7b3f7a8c9f4"}
Jul 26 15:38:26.232958 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.232849    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ptzns" event={"ID":"357d29fb-e72d-43f6-88c7-6c014abbd9c8","Type":"ContainerStarted","Data":"52772aa7f104295812d69d1827b1cd62f1973321a29eb8ab1d829e251bc95d20"}
Jul 26 15:38:26.234181 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.234147    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rx9nc" event={"ID":"ff85593b-fd58-4b0b-bfd6-8b378ca48af1","Type":"ContainerStarted","Data":"a74f12bcf80048ea107ac6f3fd80199844909093ed87be711a7577ff4d75023f"}
Jul 26 15:38:26.235495 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.235474    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5" event={"ID":"5ec8eed3-217d-4636-bb62-f5b48aa645b5","Type":"ContainerStarted","Data":"25ae7073768ae7c891d2841810723526a5b02a25d5465e9fc3199dcfa3e71ced"}
Jul 26 15:38:26.237295 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.236941    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mr2zm" event={"ID":"e15fb8a5-0274-49d5-a081-6b7097b526e1","Type":"ContainerStarted","Data":"8bff08f0007a2b7a81fa1c1ab9c9ffbb714525fc1e031011b88b6699e246083f"}
Jul 26 15:38:26.238626 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.238564    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tlcgb" event={"ID":"a8c04d11-ba9f-4142-906a-1e10f3ad0954","Type":"ContainerStarted","Data":"b238eefe67b8476a189b138b3f4f2927f65eab754b2cf4181513726b95df90e7"}
Jul 26 15:38:26.244097 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.243922    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d7sqd" podStartSLOduration=3.128876237 podStartE2EDuration="21.243904962s" totalImagesPullingTime="18.115028725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:26.243578916 +0000 UTC m=+21.742968748" watchObservedRunningTime="2026-07-26 15:38:26.243904962 +0000 UTC m=+21.743294796"
Jul 26 15:38:26.244380 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.244352    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-10.ec2.internal" podStartSLOduration=20.244342053 podStartE2EDuration="20.244342053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:10.216805111 +0000 UTC m=+5.716194951" watchObservedRunningTime="2026-07-26 15:38:26.244342053 +0000 UTC m=+21.743731905"
Jul 26 15:38:26.256372 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.256337    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5cr5p" podStartSLOduration=3.346235871 podStartE2EDuration="21.256324713s" totalImagesPullingTime="17.910088842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:26.256004077 +0000 UTC m=+21.755393914" watchObservedRunningTime="2026-07-26 15:38:26.256324713 +0000 UTC m=+21.755714559"
Jul 26 15:38:26.288699 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.288657    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tlcgb" podStartSLOduration=3.302120934 podStartE2EDuration="21.288641685s" totalImagesPullingTime="17.986520751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:26.288563072 +0000 UTC m=+21.787952922" watchObservedRunningTime="2026-07-26 15:38:26.288641685 +0000 UTC m=+21.788031526"
Jul 26 15:38:26.305601 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.305565    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ptzns" podStartSLOduration=3.354382515 podStartE2EDuration="21.305551827s" totalImagesPullingTime="17.951169312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:26.305088744 +0000 UTC m=+21.804478580" watchObservedRunningTime="2026-07-26 15:38:26.305551827 +0000 UTC m=+21.804941663"
Jul 26 15:38:26.318252 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:26.318215    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rx9nc" podStartSLOduration=3.408220064 podStartE2EDuration="21.318200998s" totalImagesPullingTime="17.909980934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:26.317632859 +0000 UTC m=+21.817022695" watchObservedRunningTime="2026-07-26 15:38:26.318200998 +0000 UTC m=+21.817590835"
Jul 26 15:38:27.242624 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.242598    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj" event={"ID":"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6","Type":"ContainerStarted","Data":"a2bb1438c7f42bdbd922a4c239f790c78ba9dbc9c494932ae103f21f455dd68b"}
Jul 26 15:38:27.243186 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.242665    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj" event={"ID":"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6","Type":"ContainerStarted","Data":"055513b86ba2d5775eecdeba26ce6d72a9af681d90447aa1236681f464855fe6"}
Jul 26 15:38:27.243186 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.242701    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj" event={"ID":"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6","Type":"ContainerStarted","Data":"2b6f4d8a665ed98f6cab97ac609c8e7e358e967b6df492799c08a8c09a408f8e"}
Jul 26 15:38:27.243186 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.242740    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj" event={"ID":"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6","Type":"ContainerStarted","Data":"2210164b828fa14f13a58b8e279de6b6c147de8cc24de38e4834e03a02a8abd4"}
Jul 26 15:38:27.243912 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.243873    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dbdsm" event={"ID":"8ead3ac8-eea8-41eb-b1a4-7198411db6bc","Type":"ContainerStarted","Data":"350dfe19a24e7cbae02c1d4d81e412014d3bec3a4cadb4b1af100910768d6366"}
Jul 26 15:38:27.245114 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.245093    2560 generic.go:356] "Generic (PLEG): container finished" podID="e15fb8a5-0274-49d5-a081-6b7097b526e1" containerID="8bff08f0007a2b7a81fa1c1ab9c9ffbb714525fc1e031011b88b6699e246083f" exitCode=0
Jul 26 15:38:27.245203 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.245184    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mr2zm" event={"ID":"e15fb8a5-0274-49d5-a081-6b7097b526e1","Type":"ContainerDied","Data":"8bff08f0007a2b7a81fa1c1ab9c9ffbb714525fc1e031011b88b6699e246083f"}
Jul 26 15:38:27.396081 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.396061    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 26 15:38:27.971931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.971809    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-26T15:38:27.396075951Z","UUID":"72325491-d632-4e11-aa1a-b30c58355f2c","Handler":null,"Name":"","Endpoint":""}
Jul 26 15:38:27.973446 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.973418    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 26 15:38:27.973576 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:27.973460    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 26 15:38:28.133180 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:28.133151    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:28.133339 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:28.133155    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:28.133339 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:28.133264    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:28.133454 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:28.133357    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:28.133454 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:28.133155    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:28.133543 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:28.133487    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:28.249983 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:28.249908    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5" event={"ID":"5ec8eed3-217d-4636-bb62-f5b48aa645b5","Type":"ContainerStarted","Data":"4bb819b2c9287d058a623f6cffb4eecc5403862679fe29cfde3e33183f61cc0f"}
Jul 26 15:38:28.263006 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:28.262936    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dbdsm" podStartSLOduration=5.284095609 podStartE2EDuration="23.262922719s" totalImagesPullingTime="17.97882711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:28.262485465 +0000 UTC m=+23.761875302" watchObservedRunningTime="2026-07-26 15:38:28.262922719 +0000 UTC m=+23.762312554"
Jul 26 15:38:29.145193 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:29.145161    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:29.145867 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:29.145848    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:29.254242 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:29.254213    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5" event={"ID":"5ec8eed3-217d-4636-bb62-f5b48aa645b5","Type":"ContainerStarted","Data":"92db1fabdf265e35d56ab1821ecb6b2c916dc2fde2cb0aebeb8a1ee22ac5222a"}
Jul 26 15:38:29.257900 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:29.257856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj" event={"ID":"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6","Type":"ContainerStarted","Data":"7084b38c152a3e8ee2b38de2a6423898d4ebfdd8711a937277cff04371683dee"}
Jul 26 15:38:29.258012 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:29.257944    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:29.258526 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:29.258508    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rx9nc"
Jul 26 15:38:29.269165 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:29.269122    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwbx5" podStartSLOduration=4.010373616 podStartE2EDuration="24.269107338s" totalImagesPullingTime="20.258733722s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:29.268584224 +0000 UTC m=+24.767974060" watchObservedRunningTime="2026-07-26 15:38:29.269107338 +0000 UTC m=+24.768497174"
Jul 26 15:38:30.132689 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:30.132657    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:30.132914 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:30.132669    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:30.132914 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:30.132787    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:30.132914 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:30.132774    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:30.132914 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:30.132856    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:30.133123 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:30.132975    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:31.266737 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:31.266709    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj" event={"ID":"2985b5a1-4ad3-451a-b8a5-1c2f65bc12a6","Type":"ContainerStarted","Data":"e1071c85d20d93c8715ab493d2cc33299489d028be570771a5a5122416a19b3e"}
Jul 26 15:38:31.267421 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:31.266970    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:31.280932 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:31.280786    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:31.288910 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:31.288859    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj" podStartSLOduration=8.283908053 podStartE2EDuration="26.288846918s" totalImagesPullingTime="18.004938865s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:31.28794814 +0000 UTC m=+26.787337976" watchObservedRunningTime="2026-07-26 15:38:31.288846918 +0000 UTC m=+26.788236754"
Jul 26 15:38:32.133440 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:32.133414    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:32.133570 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:32.133501    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:32.133570 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:32.133509    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:32.133659 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:32.133569    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:32.133659 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:32.133590    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:32.133734 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:32.133681    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:32.270007 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:32.269981    2560 generic.go:356] "Generic (PLEG): container finished" podID="e15fb8a5-0274-49d5-a081-6b7097b526e1" containerID="79a3890e24cad60092d1f0ac28a67defd61652dd0f8e71597d1f97ed70d24b7a" exitCode=0
Jul 26 15:38:32.270490 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:32.270076    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mr2zm" event={"ID":"e15fb8a5-0274-49d5-a081-6b7097b526e1","Type":"ContainerDied","Data":"79a3890e24cad60092d1f0ac28a67defd61652dd0f8e71597d1f97ed70d24b7a"}
Jul 26 15:38:32.271234 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:32.270624    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:32.271234 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:32.270695    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:32.284963 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:32.284934    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:38:33.411280 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:33.411113    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4dxwj"]
Jul 26 15:38:33.411679 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:33.411356    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:33.411679 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:33.411431    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:33.414040 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:33.413997    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lkfjb"]
Jul 26 15:38:33.414152 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:33.414097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:33.414215 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:33.414192    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:33.414602 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:33.414584    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pglx2"]
Jul 26 15:38:33.414674 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:33.414664    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:33.414755 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:33.414733    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:34.275351 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:34.275317    2560 generic.go:356] "Generic (PLEG): container finished" podID="e15fb8a5-0274-49d5-a081-6b7097b526e1" containerID="e2a88f5e456a294e9572c45a1aaa9f7b70910ecb7d483c6f50c5610354a9b81d" exitCode=0
Jul 26 15:38:34.275514 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:34.275407    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mr2zm" event={"ID":"e15fb8a5-0274-49d5-a081-6b7097b526e1","Type":"ContainerDied","Data":"e2a88f5e456a294e9572c45a1aaa9f7b70910ecb7d483c6f50c5610354a9b81d"}
Jul 26 15:38:35.134151 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:35.134088    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:35.134564 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:35.134168    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:35.134564 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:35.134194    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:35.134564 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:35.134273    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:35.134564 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:35.134299    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:35.134564 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:35.134374    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:36.281847 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:36.281812    2560 generic.go:356] "Generic (PLEG): container finished" podID="e15fb8a5-0274-49d5-a081-6b7097b526e1" containerID="6cb05c4f252a3cbba43673efa77e1b9db9c929ce8bec980e701a27b497bda7d6" exitCode=0
Jul 26 15:38:36.282410 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:36.281855    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mr2zm" event={"ID":"e15fb8a5-0274-49d5-a081-6b7097b526e1","Type":"ContainerDied","Data":"6cb05c4f252a3cbba43673efa77e1b9db9c929ce8bec980e701a27b497bda7d6"}
Jul 26 15:38:37.132679 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:37.132602    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:37.132679 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:37.132606    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:37.132877 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:37.132758    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dxwj" podUID="9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f"
Jul 26 15:38:37.132877 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:37.132795    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:37.133008 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:37.132871    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lkfjb" podUID="596c9959-6728-4c8f-8c65-faec8b068891"
Jul 26 15:38:37.133061 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:37.133005    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pglx2" podUID="89226f34-999b-4a77-a5f7-44f8c026ef55"
Jul 26 15:38:38.678232 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.678203    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:38.678643 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:38.678320    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:38.678643 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:38.678373    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs podName:89226f34-999b-4a77-a5f7-44f8c026ef55 nodeName:}" failed. No retries permitted until 2026-07-26 15:39:10.678355917 +0000 UTC m=+66.177745734 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs") pod "network-metrics-daemon-pglx2" (UID: "89226f34-999b-4a77-a5f7-44f8c026ef55") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:38.763185 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.763111    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-10.ec2.internal" event="NodeReady"
Jul 26 15:38:38.763323 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.763236    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 26 15:38:38.778731 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.778704    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s9xg\" (UniqueName: \"kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg\") pod \"network-check-target-lkfjb\" (UID: \"596c9959-6728-4c8f-8c65-faec8b068891\") " pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:38.778879 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.778739    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:38.778879 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:38.778856    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:38.779017 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:38.778879    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:38.779017 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:38.778916    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:38.779017 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:38.778926    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret podName:9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f nodeName:}" failed. No retries permitted until 2026-07-26 15:39:10.778907505 +0000 UTC m=+66.278297324 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret") pod "global-pull-secret-syncer-4dxwj" (UID: "9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:38.779017 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:38.778929    2560 projected.go:196] Error preparing data for projected volume kube-api-access-7s9xg for pod openshift-network-diagnostics/network-check-target-lkfjb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:38.779017 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:38.778980    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg podName:596c9959-6728-4c8f-8c65-faec8b068891 nodeName:}" failed. No retries permitted until 2026-07-26 15:39:10.778964534 +0000 UTC m=+66.278354365 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7s9xg" (UniqueName: "kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg") pod "network-check-target-lkfjb" (UID: "596c9959-6728-4c8f-8c65-faec8b068891") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:38.798107 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.798081    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-747679849d-sqgv2"]
Jul 26 15:38:38.801472 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.801449    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:38.803423 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.803400    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 26 15:38:38.803514 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.803428    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wk5h4\""
Jul 26 15:38:38.803601 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.803585    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 26 15:38:38.803683 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.803667    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 26 15:38:38.808442 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.808416    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-28cqg"]
Jul 26 15:38:38.808633 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.808614    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 26 15:38:38.812117 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.812099    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dssgk"]
Jul 26 15:38:38.812230 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.812214    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-28cqg"
Jul 26 15:38:38.817449 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.817003    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-747679849d-sqgv2"]
Jul 26 15:38:38.817449 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.817389    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:38.818282 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.818224    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 26 15:38:38.819113 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.819082    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 26 15:38:38.819827 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.819793    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-b8h2r\""
Jul 26 15:38:38.820794 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.820040    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 26 15:38:38.820949 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.820039    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 26 15:38:38.821112 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.820139    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 26 15:38:38.821307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.820185    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t6bwt\""
Jul 26 15:38:38.821307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.820964    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-28cqg"]
Jul 26 15:38:38.824054 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.823680    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dssgk"]
Jul 26 15:38:38.980264 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980232    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-bound-sa-token\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:38.980264 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980265    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmqk6\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-kube-api-access-gmqk6\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:38.980563 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980284    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:38.980563 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980363    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:38.980563 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980403    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/714ceae0-9812-49c6-8a81-5b64ed3bebce-tmp-dir\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:38.980563 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980429    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bcc672a2-bb64-455b-bfd9-8f5ce020365a-cert\") pod \"ingress-canary-28cqg\" (UID: \"bcc672a2-bb64-455b-bfd9-8f5ce020365a\") " pod="openshift-ingress-canary/ingress-canary-28cqg"
Jul 26 15:38:38.980744 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980561    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e3d92b7-6b62-405f-af37-0064c7feb529-ca-trust-extracted\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:38.980744 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980600    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-installation-pull-secrets\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:38.980744 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980633    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/714ceae0-9812-49c6-8a81-5b64ed3bebce-config-volume\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:38.980744 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980713    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-certificates\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:38.980938 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980746    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-trusted-ca\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:38.980938 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980766    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsbt4\" (UniqueName: \"kubernetes.io/projected/bcc672a2-bb64-455b-bfd9-8f5ce020365a-kube-api-access-gsbt4\") pod \"ingress-canary-28cqg\" (UID: \"bcc672a2-bb64-455b-bfd9-8f5ce020365a\") " pod="openshift-ingress-canary/ingress-canary-28cqg"
Jul 26 15:38:38.980938 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980795    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66ct7\" (UniqueName: \"kubernetes.io/projected/714ceae0-9812-49c6-8a81-5b64ed3bebce-kube-api-access-66ct7\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:38.980938 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:38.980823    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-image-registry-private-configuration\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.081651 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.081576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsbt4\" (UniqueName: \"kubernetes.io/projected/bcc672a2-bb64-455b-bfd9-8f5ce020365a-kube-api-access-gsbt4\") pod \"ingress-canary-28cqg\" (UID: \"bcc672a2-bb64-455b-bfd9-8f5ce020365a\") " pod="openshift-ingress-canary/ingress-canary-28cqg"
Jul 26 15:38:39.081651 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.081617    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66ct7\" (UniqueName: \"kubernetes.io/projected/714ceae0-9812-49c6-8a81-5b64ed3bebce-kube-api-access-66ct7\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:39.081651 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.081637    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-image-registry-private-configuration\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.081920 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.081673    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-bound-sa-token\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.081920 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.081854    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmqk6\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-kube-api-access-gmqk6\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.082031 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.081919    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.082031 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.081949    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:39.082031 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.081976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/714ceae0-9812-49c6-8a81-5b64ed3bebce-tmp-dir\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:39.082031 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.081998    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bcc672a2-bb64-455b-bfd9-8f5ce020365a-cert\") pod \"ingress-canary-28cqg\" (UID: \"bcc672a2-bb64-455b-bfd9-8f5ce020365a\") " pod="openshift-ingress-canary/ingress-canary-28cqg"
Jul 26 15:38:39.082211 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:39.082056    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 15:38:39.082211 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:39.082071    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 15:38:39.082211 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:39.082136    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls podName:714ceae0-9812-49c6-8a81-5b64ed3bebce nodeName:}" failed. No retries permitted until 2026-07-26 15:38:39.582115542 +0000 UTC m=+35.081505369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls") pod "dns-default-dssgk" (UID: "714ceae0-9812-49c6-8a81-5b64ed3bebce") : secret "dns-default-metrics-tls" not found
Jul 26 15:38:39.082211 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:39.082076    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-747679849d-sqgv2: secret "image-registry-tls" not found
Jul 26 15:38:39.082402 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:39.082238    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls podName:4e3d92b7-6b62-405f-af37-0064c7feb529 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:39.582218568 +0000 UTC m=+35.081608399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls") pod "image-registry-747679849d-sqgv2" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529") : secret "image-registry-tls" not found
Jul 26 15:38:39.082402 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.082078    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e3d92b7-6b62-405f-af37-0064c7feb529-ca-trust-extracted\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.082402 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.082288    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-installation-pull-secrets\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.082402 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.082298    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/714ceae0-9812-49c6-8a81-5b64ed3bebce-tmp-dir\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:39.082402 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.082318    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/714ceae0-9812-49c6-8a81-5b64ed3bebce-config-volume\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:39.082402 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.082390    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-certificates\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.082689 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.082428    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-trusted-ca\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.082689 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.082434    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e3d92b7-6b62-405f-af37-0064c7feb529-ca-trust-extracted\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.082961 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.082938    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-certificates\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.082961 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.082953    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/714ceae0-9812-49c6-8a81-5b64ed3bebce-config-volume\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:39.083294 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.083267    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-trusted-ca\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.086548 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.086415    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-image-registry-private-configuration\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.086652 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.086440    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bcc672a2-bb64-455b-bfd9-8f5ce020365a-cert\") pod \"ingress-canary-28cqg\" (UID: \"bcc672a2-bb64-455b-bfd9-8f5ce020365a\") " pod="openshift-ingress-canary/ingress-canary-28cqg"
Jul 26 15:38:39.086652 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.086479    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-installation-pull-secrets\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.090713 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.090693    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-bound-sa-token\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.090815 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.090722    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsbt4\" (UniqueName: \"kubernetes.io/projected/bcc672a2-bb64-455b-bfd9-8f5ce020365a-kube-api-access-gsbt4\") pod \"ingress-canary-28cqg\" (UID: \"bcc672a2-bb64-455b-bfd9-8f5ce020365a\") " pod="openshift-ingress-canary/ingress-canary-28cqg"
Jul 26 15:38:39.091055 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.091025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66ct7\" (UniqueName: \"kubernetes.io/projected/714ceae0-9812-49c6-8a81-5b64ed3bebce-kube-api-access-66ct7\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:39.091609 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.091589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmqk6\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-kube-api-access-gmqk6\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.129349 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.129321    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-28cqg"
Jul 26 15:38:39.133629 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.133607    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:38:39.133742 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.133647    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:38:39.133742 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.133607    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:38:39.136488 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.136222    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8j62j\""
Jul 26 15:38:39.136488 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.136330    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pkvmj\""
Jul 26 15:38:39.136488 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.136343    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 15:38:39.136488 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.136431    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 15:38:39.136742 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.136550    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 15:38:39.136742 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.136601    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 15:38:39.586783 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.586744    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:39.586783 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:39.586776    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:39.587053 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:39.586887    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 15:38:39.587053 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:39.586915    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 15:38:39.587053 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:39.586933    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-747679849d-sqgv2: secret "image-registry-tls" not found
Jul 26 15:38:39.587053 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:39.586962    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls podName:714ceae0-9812-49c6-8a81-5b64ed3bebce nodeName:}" failed. No retries permitted until 2026-07-26 15:38:40.586943673 +0000 UTC m=+36.086333491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls") pod "dns-default-dssgk" (UID: "714ceae0-9812-49c6-8a81-5b64ed3bebce") : secret "dns-default-metrics-tls" not found
Jul 26 15:38:39.587053 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:39.586993    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls podName:4e3d92b7-6b62-405f-af37-0064c7feb529 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:40.586975645 +0000 UTC m=+36.086365461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls") pod "image-registry-747679849d-sqgv2" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529") : secret "image-registry-tls" not found
Jul 26 15:38:40.479514 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:40.479491    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-28cqg"]
Jul 26 15:38:40.483460 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:40.483437    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcc672a2_bb64_455b_bfd9_8f5ce020365a.slice/crio-5e863ceb5c9806103bfd5dba0626e3b9b6a4103f9dd111405f06b9220c9e9161 WatchSource:0}: Error finding container 5e863ceb5c9806103bfd5dba0626e3b9b6a4103f9dd111405f06b9220c9e9161: Status 404 returned error can't find the container with id 5e863ceb5c9806103bfd5dba0626e3b9b6a4103f9dd111405f06b9220c9e9161
Jul 26 15:38:40.594154 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:40.594134    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:40.594215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:40.594164    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:40.594291 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:40.594273    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 15:38:40.594291 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:40.594286    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 15:38:40.594346 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:40.594295    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-747679849d-sqgv2: secret "image-registry-tls" not found
Jul 26 15:38:40.594346 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:40.594334    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls podName:714ceae0-9812-49c6-8a81-5b64ed3bebce nodeName:}" failed. No retries permitted until 2026-07-26 15:38:42.594322364 +0000 UTC m=+38.093712180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls") pod "dns-default-dssgk" (UID: "714ceae0-9812-49c6-8a81-5b64ed3bebce") : secret "dns-default-metrics-tls" not found
Jul 26 15:38:40.594409 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:40.594347    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls podName:4e3d92b7-6b62-405f-af37-0064c7feb529 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:42.594341352 +0000 UTC m=+38.093731168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls") pod "image-registry-747679849d-sqgv2" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529") : secret "image-registry-tls" not found
Jul 26 15:38:41.299598 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:41.299566    2560 generic.go:356] "Generic (PLEG): container finished" podID="e15fb8a5-0274-49d5-a081-6b7097b526e1" containerID="b59d7a7a2895af5adba2d91c1ba50999b88d0a75bcdbaf0438ae02c73321917e" exitCode=0
Jul 26 15:38:41.299770 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:41.299651    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mr2zm" event={"ID":"e15fb8a5-0274-49d5-a081-6b7097b526e1","Type":"ContainerDied","Data":"b59d7a7a2895af5adba2d91c1ba50999b88d0a75bcdbaf0438ae02c73321917e"}
Jul 26 15:38:41.303818 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:41.303762    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-28cqg" event={"ID":"bcc672a2-bb64-455b-bfd9-8f5ce020365a","Type":"ContainerStarted","Data":"5e863ceb5c9806103bfd5dba0626e3b9b6a4103f9dd111405f06b9220c9e9161"}
Jul 26 15:38:42.310025 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:42.309790    2560 generic.go:356] "Generic (PLEG): container finished" podID="e15fb8a5-0274-49d5-a081-6b7097b526e1" containerID="5f59a3055ac2aa7ca5ce3fd9be507521ab7f63fc4a63976489364cf131ff1bf7" exitCode=0
Jul 26 15:38:42.310025 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:42.309867    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mr2zm" event={"ID":"e15fb8a5-0274-49d5-a081-6b7097b526e1","Type":"ContainerDied","Data":"5f59a3055ac2aa7ca5ce3fd9be507521ab7f63fc4a63976489364cf131ff1bf7"}
Jul 26 15:38:42.611519 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:42.611492    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:42.611652 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:42.611521    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:42.611713 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:42.611658    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 15:38:42.611774 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:42.611723    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls podName:714ceae0-9812-49c6-8a81-5b64ed3bebce nodeName:}" failed. No retries permitted until 2026-07-26 15:38:46.611704704 +0000 UTC m=+42.111094534 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls") pod "dns-default-dssgk" (UID: "714ceae0-9812-49c6-8a81-5b64ed3bebce") : secret "dns-default-metrics-tls" not found
Jul 26 15:38:42.611774 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:42.611658    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 15:38:42.611877 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:42.611776    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-747679849d-sqgv2: secret "image-registry-tls" not found
Jul 26 15:38:42.611877 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:42.611840    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls podName:4e3d92b7-6b62-405f-af37-0064c7feb529 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:46.611820175 +0000 UTC m=+42.111210004 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls") pod "image-registry-747679849d-sqgv2" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529") : secret "image-registry-tls" not found
Jul 26 15:38:43.070568 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.070536    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6lcr8"]
Jul 26 15:38:43.087736 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.087661    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6lcr8"]
Jul 26 15:38:43.087866 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.087785    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.089633 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.089611    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 26 15:38:43.089772 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.089734    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fwcq5\""
Jul 26 15:38:43.090403 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.090373    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 26 15:38:43.090484 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.090460    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 26 15:38:43.090553 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.090487    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 26 15:38:43.115909 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.115175    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.115909 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.115214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.115909 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.115239    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z89d7\" (UniqueName: \"kubernetes.io/projected/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-api-access-z89d7\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.115909 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.115266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-data-volume\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.115909 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.115343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-crio-socket\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.216218 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.216181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.216218 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.216220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.216444 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.216244    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z89d7\" (UniqueName: \"kubernetes.io/projected/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-api-access-z89d7\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.216444 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:43.216304    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 15:38:43.216444 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:43.216334    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 15:38:43.216444 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.216359    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-data-volume\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.216444 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:43.216379    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm podName:9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:43.716363976 +0000 UTC m=+39.215753810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6lcr8" (UID: "9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663") : configmap "kube-rbac-proxy" not found
Jul 26 15:38:43.216444 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:43.216432    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls podName:9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:43.716417868 +0000 UTC m=+39.215807696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6lcr8" (UID: "9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663") : secret "insights-runtime-extractor-tls" not found
Jul 26 15:38:43.216695 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.216530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-crio-socket\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.216695 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.216604    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-data-volume\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.216759 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.216707    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-crio-socket\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.225567 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.225538    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z89d7\" (UniqueName: \"kubernetes.io/projected/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-api-access-z89d7\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.313473 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.313436    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-28cqg" event={"ID":"bcc672a2-bb64-455b-bfd9-8f5ce020365a","Type":"ContainerStarted","Data":"f0f22c798e918c9761bf1bb7070a0a20430a5dbb8d40860b8d58ace51da143df"}
Jul 26 15:38:43.316800 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.316775    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mr2zm" event={"ID":"e15fb8a5-0274-49d5-a081-6b7097b526e1","Type":"ContainerStarted","Data":"d921c8f8cab23355772e05c51a4db1484826112b7c2393495ccf9878499d87cf"}
Jul 26 15:38:43.327180 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.327130    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-28cqg" podStartSLOduration=2.898323491 podStartE2EDuration="5.327111617s" totalImagesPullingTime="2.428788126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:43.326650568 +0000 UTC m=+38.826040407" watchObservedRunningTime="2026-07-26 15:38:43.327111617 +0000 UTC m=+38.826501456"
Jul 26 15:38:43.346884 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.346802    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mr2zm" podStartSLOduration=10.283878007 podStartE2EDuration="38.346785159s" totalImagesPullingTime="28.062907152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:43.345535942 +0000 UTC m=+38.844925780" watchObservedRunningTime="2026-07-26 15:38:43.346785159 +0000 UTC m=+38.846175000"
Jul 26 15:38:43.437966 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.437935    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj"]
Jul 26 15:38:43.445867 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.445846    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj"
Jul 26 15:38:43.448391 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.448363    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 26 15:38:43.448391 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.448375    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-8h5dv\""
Jul 26 15:38:43.448545 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.448391    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 26 15:38:43.452617 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.452592    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj"]
Jul 26 15:38:43.519558 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.519522    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnrf5\" (UniqueName: \"kubernetes.io/projected/eed26f64-6d60-47e4-9086-053d3ccd0b0d-kube-api-access-qnrf5\") pod \"migrator-6865c9967-s92qj\" (UID: \"eed26f64-6d60-47e4-9086-053d3ccd0b0d\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj"
Jul 26 15:38:43.620964 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.620872    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnrf5\" (UniqueName: \"kubernetes.io/projected/eed26f64-6d60-47e4-9086-053d3ccd0b0d-kube-api-access-qnrf5\") pod \"migrator-6865c9967-s92qj\" (UID: \"eed26f64-6d60-47e4-9086-053d3ccd0b0d\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj"
Jul 26 15:38:43.630525 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.630504    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnrf5\" (UniqueName: \"kubernetes.io/projected/eed26f64-6d60-47e4-9086-053d3ccd0b0d-kube-api-access-qnrf5\") pod \"migrator-6865c9967-s92qj\" (UID: \"eed26f64-6d60-47e4-9086-053d3ccd0b0d\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj"
Jul 26 15:38:43.722360 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.722329    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.722505 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.722366    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:43.722505 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:43.722448    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 15:38:43.722599 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:43.722507    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm podName:9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:44.722491041 +0000 UTC m=+40.221880857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6lcr8" (UID: "9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663") : configmap "kube-rbac-proxy" not found
Jul 26 15:38:43.722599 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:43.722545    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 15:38:43.722599 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:43.722597    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls podName:9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:44.72258433 +0000 UTC m=+40.221974146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6lcr8" (UID: "9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663") : secret "insights-runtime-extractor-tls" not found
Jul 26 15:38:43.755388 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.755366    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj"
Jul 26 15:38:43.891482 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:43.891321    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj"]
Jul 26 15:38:43.893875 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:43.893850    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed26f64_6d60_47e4_9086_053d3ccd0b0d.slice/crio-986153fecb5c310174d4209c019500988afd5188e3f9f5d81008242418082c5a WatchSource:0}: Error finding container 986153fecb5c310174d4209c019500988afd5188e3f9f5d81008242418082c5a: Status 404 returned error can't find the container with id 986153fecb5c310174d4209c019500988afd5188e3f9f5d81008242418082c5a
Jul 26 15:38:44.320719 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:44.320644    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj" event={"ID":"eed26f64-6d60-47e4-9086-053d3ccd0b0d","Type":"ContainerStarted","Data":"986153fecb5c310174d4209c019500988afd5188e3f9f5d81008242418082c5a"}
Jul 26 15:38:44.673709 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:44.673676    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tlcgb_a8c04d11-ba9f-4142-906a-1e10f3ad0954/dns-node-resolver/0.log"
Jul 26 15:38:44.732213 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:44.732175    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:44.732213 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:44.732214    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:44.732403 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:44.732298    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 15:38:44.732403 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:44.732333    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 15:38:44.732403 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:44.732371    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm podName:9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:46.732349052 +0000 UTC m=+42.231738888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6lcr8" (UID: "9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663") : configmap "kube-rbac-proxy" not found
Jul 26 15:38:44.732403 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:44.732385    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls podName:9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:46.732378754 +0000 UTC m=+42.231768569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6lcr8" (UID: "9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663") : secret "insights-runtime-extractor-tls" not found
Jul 26 15:38:45.264244 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.264208    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lhmk2"]
Jul 26 15:38:45.280365 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.280341    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lhmk2"]
Jul 26 15:38:45.280500 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.280424    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.282752 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.282732    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 26 15:38:45.282878 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.282822    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 26 15:38:45.282942 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.282885    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 26 15:38:45.283339 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.283323    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-nh7td\""
Jul 26 15:38:45.283622 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.283606    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 26 15:38:45.335363 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.335341    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/13676be7-f18b-44c5-8ee3-be78c06cf828-signing-key\") pod \"service-ca-658d7b7c78-lhmk2\" (UID: \"13676be7-f18b-44c5-8ee3-be78c06cf828\") " pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.335620 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.335382    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgv76\" (UniqueName: \"kubernetes.io/projected/13676be7-f18b-44c5-8ee3-be78c06cf828-kube-api-access-fgv76\") pod \"service-ca-658d7b7c78-lhmk2\" (UID: \"13676be7-f18b-44c5-8ee3-be78c06cf828\") " pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.335620 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.335401    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/13676be7-f18b-44c5-8ee3-be78c06cf828-signing-cabundle\") pod \"service-ca-658d7b7c78-lhmk2\" (UID: \"13676be7-f18b-44c5-8ee3-be78c06cf828\") " pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.436023 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.436000    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/13676be7-f18b-44c5-8ee3-be78c06cf828-signing-key\") pod \"service-ca-658d7b7c78-lhmk2\" (UID: \"13676be7-f18b-44c5-8ee3-be78c06cf828\") " pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.436121 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.436047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgv76\" (UniqueName: \"kubernetes.io/projected/13676be7-f18b-44c5-8ee3-be78c06cf828-kube-api-access-fgv76\") pod \"service-ca-658d7b7c78-lhmk2\" (UID: \"13676be7-f18b-44c5-8ee3-be78c06cf828\") " pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.436121 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.436065    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/13676be7-f18b-44c5-8ee3-be78c06cf828-signing-cabundle\") pod \"service-ca-658d7b7c78-lhmk2\" (UID: \"13676be7-f18b-44c5-8ee3-be78c06cf828\") " pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.436628 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.436612    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/13676be7-f18b-44c5-8ee3-be78c06cf828-signing-cabundle\") pod \"service-ca-658d7b7c78-lhmk2\" (UID: \"13676be7-f18b-44c5-8ee3-be78c06cf828\") " pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.438267 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.438248    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/13676be7-f18b-44c5-8ee3-be78c06cf828-signing-key\") pod \"service-ca-658d7b7c78-lhmk2\" (UID: \"13676be7-f18b-44c5-8ee3-be78c06cf828\") " pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.443122 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.443087    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgv76\" (UniqueName: \"kubernetes.io/projected/13676be7-f18b-44c5-8ee3-be78c06cf828-kube-api-access-fgv76\") pod \"service-ca-658d7b7c78-lhmk2\" (UID: \"13676be7-f18b-44c5-8ee3-be78c06cf828\") " pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.478528 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.478512    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5cr5p_a53b35cf-9867-4dd7-9e53-1cbab673f3fe/node-ca/0.log"
Jul 26 15:38:45.588764 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.588744    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2"
Jul 26 15:38:45.724630 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:45.724606    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lhmk2"]
Jul 26 15:38:45.727203 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:45.727181    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13676be7_f18b_44c5_8ee3_be78c06cf828.slice/crio-60ef8650ee98b850870817b728c6d041ff6f94898cc90d908f99d1d41012f599 WatchSource:0}: Error finding container 60ef8650ee98b850870817b728c6d041ff6f94898cc90d908f99d1d41012f599: Status 404 returned error can't find the container with id 60ef8650ee98b850870817b728c6d041ff6f94898cc90d908f99d1d41012f599
Jul 26 15:38:46.275364 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:46.275330    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-28cqg_bcc672a2-bb64-455b-bfd9-8f5ce020365a/serve-healthcheck-canary/0.log"
Jul 26 15:38:46.327628 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:46.327594    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj" event={"ID":"eed26f64-6d60-47e4-9086-053d3ccd0b0d","Type":"ContainerStarted","Data":"a7e70b36690bfa42b8ce7beb2d569ff41607e354ad688f2edcca0fade9872a46"}
Jul 26 15:38:46.327786 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:46.327653    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj" event={"ID":"eed26f64-6d60-47e4-9086-053d3ccd0b0d","Type":"ContainerStarted","Data":"a3866afbc291b71021d2ac73f6a4fc62bc9a92ffdf0c5d237648d4bd64b1e91b"}
Jul 26 15:38:46.328943 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:46.328918    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2" event={"ID":"13676be7-f18b-44c5-8ee3-be78c06cf828","Type":"ContainerStarted","Data":"60ef8650ee98b850870817b728c6d041ff6f94898cc90d908f99d1d41012f599"}
Jul 26 15:38:46.343558 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:46.343481    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s92qj" podStartSLOduration=1.7594714759999999 podStartE2EDuration="3.343464491s" totalImagesPullingTime="1.583993015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:46.343017076 +0000 UTC m=+41.842406916" watchObservedRunningTime="2026-07-26 15:38:46.343464491 +0000 UTC m=+41.842854329"
Jul 26 15:38:46.646299 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:46.646263    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:46.646299 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:46.646303    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:46.646507 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:46.646411    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 15:38:46.646507 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:46.646418    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 26 15:38:46.646507 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:46.646440    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-747679849d-sqgv2: secret "image-registry-tls" not found
Jul 26 15:38:46.646507 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:46.646473    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls podName:714ceae0-9812-49c6-8a81-5b64ed3bebce nodeName:}" failed. No retries permitted until 2026-07-26 15:38:54.64645624 +0000 UTC m=+50.145846056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls") pod "dns-default-dssgk" (UID: "714ceae0-9812-49c6-8a81-5b64ed3bebce") : secret "dns-default-metrics-tls" not found
Jul 26 15:38:46.646507 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:46.646489    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls podName:4e3d92b7-6b62-405f-af37-0064c7feb529 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:54.646483031 +0000 UTC m=+50.145872847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls") pod "image-registry-747679849d-sqgv2" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529") : secret "image-registry-tls" not found
Jul 26 15:38:46.746931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:46.746904    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:46.747052 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:46.746943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:46.747105 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:46.747085    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 15:38:46.747150 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:46.747140    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm podName:9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:50.747120987 +0000 UTC m=+46.246510820 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6lcr8" (UID: "9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663") : configmap "kube-rbac-proxy" not found
Jul 26 15:38:46.747213 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:46.747194    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 15:38:46.747264 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:46.747255    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls podName:9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:50.74724072 +0000 UTC m=+46.246630541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6lcr8" (UID: "9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663") : secret "insights-runtime-extractor-tls" not found
Jul 26 15:38:48.335580 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:48.335498    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2" event={"ID":"13676be7-f18b-44c5-8ee3-be78c06cf828","Type":"ContainerStarted","Data":"ff8a1ef3af52ff6aa4298acc82995a404202c25450aeb9ddcae2b588e91186e6"}
Jul 26 15:38:48.350169 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:48.350128    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-lhmk2" podStartSLOduration=1.1172519379999999 podStartE2EDuration="3.350113128s" totalImagesPullingTime="2.23286119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:48.349449877 +0000 UTC m=+43.848839715" watchObservedRunningTime="2026-07-26 15:38:48.350113128 +0000 UTC m=+43.849502965"
Jul 26 15:38:50.777423 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:50.777363    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:50.777927 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:50.777429    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:50.777927 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:50.777594    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 15:38:50.777927 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:38:50.777665    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls podName:9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:58.777647505 +0000 UTC m=+54.277037325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6lcr8" (UID: "9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663") : secret "insights-runtime-extractor-tls" not found
Jul 26 15:38:50.778060 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:50.778038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:54.709464 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:54.709427    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:54.709955 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:54.709473    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:54.711758 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:54.711732    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/714ceae0-9812-49c6-8a81-5b64ed3bebce-metrics-tls\") pod \"dns-default-dssgk\" (UID: \"714ceae0-9812-49c6-8a81-5b64ed3bebce\") " pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:54.711857 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:54.711771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls\") pod \"image-registry-747679849d-sqgv2\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") " pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:54.712647 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:54.712624    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:54.735316 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:54.735299    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:54.890471 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:54.890440    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-747679849d-sqgv2"]
Jul 26 15:38:54.893357 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:54.893331    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e3d92b7_6b62_405f_af37_0064c7feb529.slice/crio-8dc78dfa0f9fc2241a7b1c9eccde78657dc801f6c6405b0a43c54f23cb3249d4 WatchSource:0}: Error finding container 8dc78dfa0f9fc2241a7b1c9eccde78657dc801f6c6405b0a43c54f23cb3249d4: Status 404 returned error can't find the container with id 8dc78dfa0f9fc2241a7b1c9eccde78657dc801f6c6405b0a43c54f23cb3249d4
Jul 26 15:38:54.897714 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:54.897690    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dssgk"]
Jul 26 15:38:54.900313 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:54.900290    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714ceae0_9812_49c6_8a81_5b64ed3bebce.slice/crio-132825894ad65978cfca8115f5a5d9b23d0c0f47805ab695784bb8740776719d WatchSource:0}: Error finding container 132825894ad65978cfca8115f5a5d9b23d0c0f47805ab695784bb8740776719d: Status 404 returned error can't find the container with id 132825894ad65978cfca8115f5a5d9b23d0c0f47805ab695784bb8740776719d
Jul 26 15:38:55.353041 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:55.352932    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-747679849d-sqgv2" event={"ID":"4e3d92b7-6b62-405f-af37-0064c7feb529","Type":"ContainerStarted","Data":"f13ecfb0a6e1a66c4507a5e7a88fc50b9cf58369dcd6ad33b6a02aee135a4656"}
Jul 26 15:38:55.353041 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:55.353003    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-747679849d-sqgv2" event={"ID":"4e3d92b7-6b62-405f-af37-0064c7feb529","Type":"ContainerStarted","Data":"8dc78dfa0f9fc2241a7b1c9eccde78657dc801f6c6405b0a43c54f23cb3249d4"}
Jul 26 15:38:55.353041 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:55.353031    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:38:55.354190 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:55.354164    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dssgk" event={"ID":"714ceae0-9812-49c6-8a81-5b64ed3bebce","Type":"ContainerStarted","Data":"132825894ad65978cfca8115f5a5d9b23d0c0f47805ab695784bb8740776719d"}
Jul 26 15:38:55.370688 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:55.370640    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-747679849d-sqgv2" podStartSLOduration=34.370626542 podStartE2EDuration="34.370626542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:55.369490447 +0000 UTC m=+50.868880286" watchObservedRunningTime="2026-07-26 15:38:55.370626542 +0000 UTC m=+50.870016379"
Jul 26 15:38:57.362763 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:57.362734    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dssgk" event={"ID":"714ceae0-9812-49c6-8a81-5b64ed3bebce","Type":"ContainerStarted","Data":"fde564e99773ac4a657fc962de5a5a280cba83c3677a9343ea72f6139ff1a8be"}
Jul 26 15:38:58.371324 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:58.371293    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dssgk" event={"ID":"714ceae0-9812-49c6-8a81-5b64ed3bebce","Type":"ContainerStarted","Data":"81d192af55e19fc53d5f0a15fdc10f53c44e5067a9bb9d72962d72dfd33f48ad"}
Jul 26 15:38:58.371704 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:58.371438    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dssgk"
Jul 26 15:38:58.386135 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:58.386099    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dssgk" podStartSLOduration=18.121706389 podStartE2EDuration="20.386086436s" totalImagesPullingTime="2.264380047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:58.385238623 +0000 UTC m=+53.884628462" watchObservedRunningTime="2026-07-26 15:38:58.386086436 +0000 UTC m=+53.885476292"
Jul 26 15:38:58.844059 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:58.844021    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:58.846315 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:58.846288    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6lcr8\" (UID: \"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663\") " pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:58.998559 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:58.998529    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6lcr8"
Jul 26 15:38:59.116446 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:59.116372    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6lcr8"]
Jul 26 15:38:59.119587 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:38:59.119559    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1b35c0_c87e_4b1a_8405_e5c0fb5bb663.slice/crio-a291c16d33ce30ab62cebccdec8d681b785e5183313c1b11737f7399f930da46 WatchSource:0}: Error finding container a291c16d33ce30ab62cebccdec8d681b785e5183313c1b11737f7399f930da46: Status 404 returned error can't find the container with id a291c16d33ce30ab62cebccdec8d681b785e5183313c1b11737f7399f930da46
Jul 26 15:38:59.376128 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:38:59.376050    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6lcr8" event={"ID":"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663","Type":"ContainerStarted","Data":"a291c16d33ce30ab62cebccdec8d681b785e5183313c1b11737f7399f930da46"}
Jul 26 15:39:04.285609 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:04.285547    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sd6zj"
Jul 26 15:39:04.395804 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:04.395751    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6lcr8" event={"ID":"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663","Type":"ContainerStarted","Data":"48a288c382a9268216cbd441c97cd8591872a5db3b7a1397e7492b38e71a56ae"}
Jul 26 15:39:08.377983 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:08.377949    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dssgk"
Jul 26 15:39:08.410424 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:08.410390    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6lcr8" event={"ID":"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663","Type":"ContainerStarted","Data":"f9e4148d8ed51a267d909d8b307911e26293d9bdb72d4488d0561f9fb4ee2f0b"}
Jul 26 15:39:09.330299 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.330265    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"]
Jul 26 15:39:09.335026 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.334996    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"]
Jul 26 15:39:09.335299 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.335276    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"
Jul 26 15:39:09.337655 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.337616    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 26 15:39:09.337770 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.337682    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 26 15:39:09.338245 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.338225    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7vg9n\""
Jul 26 15:39:09.338487 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.338435    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 26 15:39:09.338582 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.338551    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.338645 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.338624    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 26 15:39:09.340673 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.340453    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 26 15:39:09.340673 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.340509    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 26 15:39:09.340673 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.340517    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 26 15:39:09.340880 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.340678    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 26 15:39:09.342621 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.342578    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"]
Jul 26 15:39:09.345580 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.345560    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"]
Jul 26 15:39:09.411244 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.411220    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-747679849d-sqgv2"]
Jul 26 15:39:09.433777 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.433754    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-hub\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.433927 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.433804    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wllwb\" (UniqueName: \"kubernetes.io/projected/f88def5a-9bf7-4efa-b359-b62f303dfa14-kube-api-access-wllwb\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.433927 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.433860    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/24b571b0-8522-4bd4-94e3-0ff2a0cfc464-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55c85b7774-z268x\" (UID: \"24b571b0-8522-4bd4-94e3-0ff2a0cfc464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"
Jul 26 15:39:09.434032 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.433947    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-ca\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.434032 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.434010    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88def5a-9bf7-4efa-b359-b62f303dfa14-additional-service-ca\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.434118 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.434034    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.434118 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.434058    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2hm5\" (UniqueName: \"kubernetes.io/projected/24b571b0-8522-4bd4-94e3-0ff2a0cfc464-kube-api-access-b2hm5\") pod \"managed-serviceaccount-addon-agent-55c85b7774-z268x\" (UID: \"24b571b0-8522-4bd4-94e3-0ff2a0cfc464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"
Jul 26 15:39:09.434118 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.434082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.524454 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.524426    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"]
Jul 26 15:39:09.530096 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.530072    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.534943 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.534922    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/24b571b0-8522-4bd4-94e3-0ff2a0cfc464-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55c85b7774-z268x\" (UID: \"24b571b0-8522-4bd4-94e3-0ff2a0cfc464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"
Jul 26 15:39:09.535055 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.534978    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-ca\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.535055 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.535019    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88def5a-9bf7-4efa-b359-b62f303dfa14-additional-service-ca\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.535055 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.535048    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.535215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.535070    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2hm5\" (UniqueName: \"kubernetes.io/projected/24b571b0-8522-4bd4-94e3-0ff2a0cfc464-kube-api-access-b2hm5\") pod \"managed-serviceaccount-addon-agent-55c85b7774-z268x\" (UID: \"24b571b0-8522-4bd4-94e3-0ff2a0cfc464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"
Jul 26 15:39:09.535215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.535092    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.535215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.535127    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-hub\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.535215 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.535181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wllwb\" (UniqueName: \"kubernetes.io/projected/f88def5a-9bf7-4efa-b359-b62f303dfa14-kube-api-access-wllwb\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.535851 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.535829    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88def5a-9bf7-4efa-b359-b62f303dfa14-additional-service-ca\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.538049 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.538025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.538150 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.538044    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-ca\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.538150 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.538135    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-hub\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.538266 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.538245    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/24b571b0-8522-4bd4-94e3-0ff2a0cfc464-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55c85b7774-z268x\" (UID: \"24b571b0-8522-4bd4-94e3-0ff2a0cfc464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"
Jul 26 15:39:09.538477 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.538454    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f88def5a-9bf7-4efa-b359-b62f303dfa14-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.541160 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.541138    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"]
Jul 26 15:39:09.545017 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.544973    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2hm5\" (UniqueName: \"kubernetes.io/projected/24b571b0-8522-4bd4-94e3-0ff2a0cfc464-kube-api-access-b2hm5\") pod \"managed-serviceaccount-addon-agent-55c85b7774-z268x\" (UID: \"24b571b0-8522-4bd4-94e3-0ff2a0cfc464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"
Jul 26 15:39:09.545141 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.545000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wllwb\" (UniqueName: \"kubernetes.io/projected/f88def5a-9bf7-4efa-b359-b62f303dfa14-kube-api-access-wllwb\") pod \"cluster-proxy-proxy-agent-686cd7c897-2bc56\" (UID: \"f88def5a-9bf7-4efa-b359-b62f303dfa14\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.635732 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.635654    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-ca-trust-extracted\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.635732 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.635690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krr2j\" (UniqueName: \"kubernetes.io/projected/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-kube-api-access-krr2j\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.635732 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.635726    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-bound-sa-token\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.636004 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.635777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-trusted-ca\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.636004 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.635824    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-installation-pull-secrets\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.636004 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.635850    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-image-registry-private-configuration\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.636004 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.635930    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-registry-certificates\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.636004 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.635958    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-registry-tls\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.657818 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.657792    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"
Jul 26 15:39:09.664296 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.664278    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"
Jul 26 15:39:09.736524 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.736494    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-ca-trust-extracted\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.736646 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.736532    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krr2j\" (UniqueName: \"kubernetes.io/projected/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-kube-api-access-krr2j\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.736646 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.736566    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-bound-sa-token\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.736745 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.736693    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-trusted-ca\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.736797 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.736755    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-installation-pull-secrets\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.736797 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.736786    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-image-registry-private-configuration\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.736919 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.736812    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-registry-certificates\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.736919 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.736846    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-registry-tls\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.736919 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.736853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-ca-trust-extracted\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.737667 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.737637    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-registry-certificates\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.737780 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.737759    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-trusted-ca\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.739524 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.739499    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-image-registry-private-configuration\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.739678 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.739654    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-installation-pull-secrets\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.739768 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.739730    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-registry-tls\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.745649 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.745629    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-bound-sa-token\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.745909 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.745868    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krr2j\" (UniqueName: \"kubernetes.io/projected/eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1-kube-api-access-krr2j\") pod \"image-registry-7ddc4ddbcf-tqgkr\" (UID: \"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1\") " pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:09.861652 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:09.861619    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:10.744956 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.744916    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:39:10.747053 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.747030    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 15:39:10.757861 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.757841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89226f34-999b-4a77-a5f7-44f8c026ef55-metrics-certs\") pod \"network-metrics-daemon-pglx2\" (UID: \"89226f34-999b-4a77-a5f7-44f8c026ef55\") " pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:39:10.845299 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.845269    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s9xg\" (UniqueName: \"kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg\") pod \"network-check-target-lkfjb\" (UID: \"596c9959-6728-4c8f-8c65-faec8b068891\") " pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:39:10.845420 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.845304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:39:10.847349 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.847327    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 15:39:10.847426 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.847341    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 15:39:10.857755 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.857731    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 15:39:10.858007 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.857991    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f-original-pull-secret\") pod \"global-pull-secret-syncer-4dxwj\" (UID: \"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f\") " pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:39:10.868612 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.868593    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s9xg\" (UniqueName: \"kubernetes.io/projected/596c9959-6728-4c8f-8c65-faec8b068891-kube-api-access-7s9xg\") pod \"network-check-target-lkfjb\" (UID: \"596c9959-6728-4c8f-8c65-faec8b068891\") " pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:39:10.947747 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.947498    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8j62j\""
Jul 26 15:39:10.955700 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.955519    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pglx2"
Jul 26 15:39:10.957319 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.957106    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pkvmj\""
Jul 26 15:39:10.958575 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.958456    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dxwj"
Jul 26 15:39:10.965720 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:10.965689    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:39:11.032923 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.032751    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x"]
Jul 26 15:39:11.065290 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.064355    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56"]
Jul 26 15:39:11.069947 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:39:11.069915    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf88def5a_9bf7_4efa_b359_b62f303dfa14.slice/crio-8e07168f36c5566b8dfdd684f19028db5bd7ed6d06f3bd400453fe8f6cfac91b WatchSource:0}: Error finding container 8e07168f36c5566b8dfdd684f19028db5bd7ed6d06f3bd400453fe8f6cfac91b: Status 404 returned error can't find the container with id 8e07168f36c5566b8dfdd684f19028db5bd7ed6d06f3bd400453fe8f6cfac91b
Jul 26 15:39:11.071855 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.071816    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"]
Jul 26 15:39:11.076506 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:39:11.076480    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebc47af_9e56_4e72_bf4f_7cdf69a1c9d1.slice/crio-4ce43c9e3c15b73c699e0643cf1e66103871f76215e01dd192b357e5cdfca9d7 WatchSource:0}: Error finding container 4ce43c9e3c15b73c699e0643cf1e66103871f76215e01dd192b357e5cdfca9d7: Status 404 returned error can't find the container with id 4ce43c9e3c15b73c699e0643cf1e66103871f76215e01dd192b357e5cdfca9d7
Jul 26 15:39:11.140843 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.140815    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pglx2"]
Jul 26 15:39:11.145351 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:39:11.145326    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89226f34_999b_4a77_a5f7_44f8c026ef55.slice/crio-a641b50d5399761b1af23a0fe96e82e0e7b551b83149592e09cbe8bef8def3d5 WatchSource:0}: Error finding container a641b50d5399761b1af23a0fe96e82e0e7b551b83149592e09cbe8bef8def3d5: Status 404 returned error can't find the container with id a641b50d5399761b1af23a0fe96e82e0e7b551b83149592e09cbe8bef8def3d5
Jul 26 15:39:11.161249 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.161223    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lkfjb"]
Jul 26 15:39:11.163801 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:39:11.163779    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596c9959_6728_4c8f_8c65_faec8b068891.slice/crio-dbdd0da5a2222c481919905a5d027b9c1e4d9b1a7a59fb94d280e1d8568ffdf1 WatchSource:0}: Error finding container dbdd0da5a2222c481919905a5d027b9c1e4d9b1a7a59fb94d280e1d8568ffdf1: Status 404 returned error can't find the container with id dbdd0da5a2222c481919905a5d027b9c1e4d9b1a7a59fb94d280e1d8568ffdf1
Jul 26 15:39:11.167358 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.167336    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4dxwj"]
Jul 26 15:39:11.170396 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:39:11.170375    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2d7ff0_73eb_4b22_a8f5_c595d7bb050f.slice/crio-d51f8505068a1b3e9ce46f55312aeb1370ddf1cace26eae55bcb0211580a176e WatchSource:0}: Error finding container d51f8505068a1b3e9ce46f55312aeb1370ddf1cace26eae55bcb0211580a176e: Status 404 returned error can't find the container with id d51f8505068a1b3e9ce46f55312aeb1370ddf1cace26eae55bcb0211580a176e
Jul 26 15:39:11.423540 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.423501    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4dxwj" event={"ID":"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f","Type":"ContainerStarted","Data":"d51f8505068a1b3e9ce46f55312aeb1370ddf1cace26eae55bcb0211580a176e"}
Jul 26 15:39:11.424695 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.424662    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x" event={"ID":"24b571b0-8522-4bd4-94e3-0ff2a0cfc464","Type":"ContainerStarted","Data":"907ca67ed759128cd63d3d8f84815da823dde6c004ce5e60615af34b0aa3684f"}
Jul 26 15:39:11.426675 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.426641    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6lcr8" event={"ID":"9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663","Type":"ContainerStarted","Data":"eafd00c0b9e59c610b80c58f74d48378a09c865028a8eb4a2ae7cf2a0efaae92"}
Jul 26 15:39:11.428071 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.428043    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lkfjb" event={"ID":"596c9959-6728-4c8f-8c65-faec8b068891","Type":"ContainerStarted","Data":"dbdd0da5a2222c481919905a5d027b9c1e4d9b1a7a59fb94d280e1d8568ffdf1"}
Jul 26 15:39:11.429152 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.429122    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pglx2" event={"ID":"89226f34-999b-4a77-a5f7-44f8c026ef55","Type":"ContainerStarted","Data":"a641b50d5399761b1af23a0fe96e82e0e7b551b83149592e09cbe8bef8def3d5"}
Jul 26 15:39:11.430315 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.430293    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56" event={"ID":"f88def5a-9bf7-4efa-b359-b62f303dfa14","Type":"ContainerStarted","Data":"8e07168f36c5566b8dfdd684f19028db5bd7ed6d06f3bd400453fe8f6cfac91b"}
Jul 26 15:39:11.431651 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.431627    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr" event={"ID":"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1","Type":"ContainerStarted","Data":"52a3686efad75c6606bc7eb8a45e762534951640c12dff0da33f4d3d4ad3d6e0"}
Jul 26 15:39:11.431751 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.431668    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr" event={"ID":"eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1","Type":"ContainerStarted","Data":"4ce43c9e3c15b73c699e0643cf1e66103871f76215e01dd192b357e5cdfca9d7"}
Jul 26 15:39:11.431808 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.431764    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:11.442226 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.442177    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6lcr8" podStartSLOduration=16.728496151999998 podStartE2EDuration="28.442164291s" totalImagesPullingTime="11.713668139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:11.441345811 +0000 UTC m=+66.940735649" watchObservedRunningTime="2026-07-26 15:39:11.442164291 +0000 UTC m=+66.941554129"
Jul 26 15:39:11.463832 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:11.463791    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr" podStartSLOduration=2.46377951 podStartE2EDuration="2.46377951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:11.463320105 +0000 UTC m=+66.962709957" watchObservedRunningTime="2026-07-26 15:39:11.46377951 +0000 UTC m=+66.963169374"
Jul 26 15:39:13.461567 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:13.460715    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pglx2" event={"ID":"89226f34-999b-4a77-a5f7-44f8c026ef55","Type":"ContainerStarted","Data":"b4e5c217e5f2f460df6b68d389fd0ac6bf4278d17fc95f6286fc8d19f6875822"}
Jul 26 15:39:13.461567 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:13.460782    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pglx2" event={"ID":"89226f34-999b-4a77-a5f7-44f8c026ef55","Type":"ContainerStarted","Data":"797cc338771e49850a49045e9bcf8382dffa613b8499a92c668e936e8ee12743"}
Jul 26 15:39:13.477844 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:13.476796    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pglx2" podStartSLOduration=67.171641181 podStartE2EDuration="1m8.476778048s" totalImagesPullingTime="1.305136867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:13.475485887 +0000 UTC m=+68.974875730" watchObservedRunningTime="2026-07-26 15:39:13.476778048 +0000 UTC m=+68.976167887"
Jul 26 15:39:19.416576 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.416547    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:39:19.485350 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.485320    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lkfjb" event={"ID":"596c9959-6728-4c8f-8c65-faec8b068891","Type":"ContainerStarted","Data":"fcdef9a0eea3c8098d507d36f98d8bdf994d707c579a38b9a673b9510257e96d"}
Jul 26 15:39:19.485504 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.485448    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:39:19.487442 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.487421    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56" event={"ID":"f88def5a-9bf7-4efa-b359-b62f303dfa14","Type":"ContainerStarted","Data":"d5f0e983db4b0283a3ca2fc02f87493576d7452ce20da6a96e74d960186f2dd1"}
Jul 26 15:39:19.487529 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.487450    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56" event={"ID":"f88def5a-9bf7-4efa-b359-b62f303dfa14","Type":"ContainerStarted","Data":"7801612c4394f9bfc111ce463a6415ed8c78ebee8fc5081590d4055f5b7d5a8a"}
Jul 26 15:39:19.487529 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.487462    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56" event={"ID":"f88def5a-9bf7-4efa-b359-b62f303dfa14","Type":"ContainerStarted","Data":"d696adeee291073d78a42baf2ad868aac9c1c6b89d0918281b0f4e1bbd1818b6"}
Jul 26 15:39:19.488788 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.488767    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4dxwj" event={"ID":"9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f","Type":"ContainerStarted","Data":"7ae29a59b920750d1c8c56ef24cefc54616ff7bf239037910f3b3fee5e56f511"}
Jul 26 15:39:19.490136 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.490116    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x" event={"ID":"24b571b0-8522-4bd4-94e3-0ff2a0cfc464","Type":"ContainerStarted","Data":"c6bd0f4ad0c0587502aa6dbf2ff5470549dfbcf1ba63ed0adcbcfe83fe539422"}
Jul 26 15:39:19.503156 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.503114    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lkfjb" podStartSLOduration=66.939988219 podStartE2EDuration="1m14.503100365s" totalImagesPullingTime="7.563112146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:19.501618862 +0000 UTC m=+75.001008700" watchObservedRunningTime="2026-07-26 15:39:19.503100365 +0000 UTC m=+75.002490209"
Jul 26 15:39:19.519744 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.519708    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-686cd7c897-2bc56" podStartSLOduration=2.857841503 podStartE2EDuration="10.519695222s" totalImagesPullingTime="7.661853719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:19.519299879 +0000 UTC m=+75.018689726" watchObservedRunningTime="2026-07-26 15:39:19.519695222 +0000 UTC m=+75.019085061"
Jul 26 15:39:19.532001 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.531958    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55c85b7774-z268x" podStartSLOduration=2.855162956 podStartE2EDuration="10.531942395s" totalImagesPullingTime="7.676779439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:19.531636448 +0000 UTC m=+75.031026288" watchObservedRunningTime="2026-07-26 15:39:19.531942395 +0000 UTC m=+75.031332234"
Jul 26 15:39:19.550535 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:19.550501    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4dxwj" podStartSLOduration=66.996874618 podStartE2EDuration="1m14.550485761s" totalImagesPullingTime="7.553611143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:19.55018411 +0000 UTC m=+75.049573961" watchObservedRunningTime="2026-07-26 15:39:19.550485761 +0000 UTC m=+75.049875601"
Jul 26 15:39:29.739723 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.739580    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-27c6h"]
Jul 26 15:39:29.743057 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.743034    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.744935 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.744912    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 26 15:39:29.745188 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.745162    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5fmmc\""
Jul 26 15:39:29.745296 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.745265    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 26 15:39:29.745357 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.745167    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 26 15:39:29.745515 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.745499    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 26 15:39:29.745607 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.745591    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 26 15:39:29.745783 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.745665    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 26 15:39:29.894764 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.894740    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.894764 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.894769    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1ea955f-7f82-42d1-9326-606883aa4bed-root\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.895003 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.894788    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-wtmp\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.895003 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.894804    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-tls\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.895003 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.894868    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxdmq\" (UniqueName: \"kubernetes.io/projected/e1ea955f-7f82-42d1-9326-606883aa4bed-kube-api-access-fxdmq\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.895003 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.894903    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-accelerators-collector-config\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.895003 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.894926    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1ea955f-7f82-42d1-9326-606883aa4bed-sys\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.895003 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.894946    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ea955f-7f82-42d1-9326-606883aa4bed-metrics-client-ca\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.895003 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.894963    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-textfile\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.995663 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1ea955f-7f82-42d1-9326-606883aa4bed-root\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.995663 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995629    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-wtmp\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.995844 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1ea955f-7f82-42d1-9326-606883aa4bed-root\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.995844 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995738    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-tls\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.995844 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995764    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-wtmp\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.995844 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995806    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxdmq\" (UniqueName: \"kubernetes.io/projected/e1ea955f-7f82-42d1-9326-606883aa4bed-kube-api-access-fxdmq\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.996051 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995850    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-accelerators-collector-config\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.996051 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995877    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1ea955f-7f82-42d1-9326-606883aa4bed-sys\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.996051 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995926    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ea955f-7f82-42d1-9326-606883aa4bed-metrics-client-ca\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.996051 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995957    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-textfile\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.996051 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.995999    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.996310 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.996121    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1ea955f-7f82-42d1-9326-606883aa4bed-sys\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.996684 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.996651    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-accelerators-collector-config\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.996744 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.996666    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ea955f-7f82-42d1-9326-606883aa4bed-metrics-client-ca\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.996886 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.996864    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-textfile\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.998317 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.998294    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:29.999074 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:29.999056    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1ea955f-7f82-42d1-9326-606883aa4bed-node-exporter-tls\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:30.002525 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:30.002501    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxdmq\" (UniqueName: \"kubernetes.io/projected/e1ea955f-7f82-42d1-9326-606883aa4bed-kube-api-access-fxdmq\") pod \"node-exporter-27c6h\" (UID: \"e1ea955f-7f82-42d1-9326-606883aa4bed\") " pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:30.054719 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:30.054697    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-27c6h"
Jul 26 15:39:30.065396 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:39:30.065366    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ea955f_7f82_42d1_9326_606883aa4bed.slice/crio-fa6dcab984593a6620b7ab01d97d385f3b0b2c276164c6bb5a21d3f1d724bfa3 WatchSource:0}: Error finding container fa6dcab984593a6620b7ab01d97d385f3b0b2c276164c6bb5a21d3f1d724bfa3: Status 404 returned error can't find the container with id fa6dcab984593a6620b7ab01d97d385f3b0b2c276164c6bb5a21d3f1d724bfa3
Jul 26 15:39:30.532945 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:30.532880    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27c6h" event={"ID":"e1ea955f-7f82-42d1-9326-606883aa4bed","Type":"ContainerStarted","Data":"fa6dcab984593a6620b7ab01d97d385f3b0b2c276164c6bb5a21d3f1d724bfa3"}
Jul 26 15:39:31.537598 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:31.537564    2560 generic.go:356] "Generic (PLEG): container finished" podID="e1ea955f-7f82-42d1-9326-606883aa4bed" containerID="4fd30b116926bb0f881da685871e40ee4b67611dc0fb2451b9d352ec4de99086" exitCode=0
Jul 26 15:39:31.537961 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:31.537648    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27c6h" event={"ID":"e1ea955f-7f82-42d1-9326-606883aa4bed","Type":"ContainerDied","Data":"4fd30b116926bb0f881da685871e40ee4b67611dc0fb2451b9d352ec4de99086"}
Jul 26 15:39:32.448523 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:32.448495    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7ddc4ddbcf-tqgkr"
Jul 26 15:39:32.543151 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:32.543116    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27c6h" event={"ID":"e1ea955f-7f82-42d1-9326-606883aa4bed","Type":"ContainerStarted","Data":"3fe411c710bc87684466802672a3c7811c532d157200da31558ea0b9574757c7"}
Jul 26 15:39:32.543593 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:32.543173    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27c6h" event={"ID":"e1ea955f-7f82-42d1-9326-606883aa4bed","Type":"ContainerStarted","Data":"aa258fd464f82362ad4d4ff378c393d2271bcc38124bb72977daa2711d710896"}
Jul 26 15:39:32.562733 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:32.562693    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-27c6h" podStartSLOduration=2.577497662 podStartE2EDuration="3.562678237s" totalImagesPullingTime="922.501617ms" totalInitContainerRuntime="62.678958ms" isStatefulPod=true podCreationTimestamp="2026-07-26 15:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:32.561430049 +0000 UTC m=+88.060819897" watchObservedRunningTime="2026-07-26 15:39:32.562678237 +0000 UTC m=+88.062068101"
Jul 26 15:39:34.430562 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.430518    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-747679849d-sqgv2" podUID="4e3d92b7-6b62-405f-af37-0064c7feb529" containerName="registry" containerID="cri-o://f13ecfb0a6e1a66c4507a5e7a88fc50b9cf58369dcd6ad33b6a02aee135a4656" gracePeriod=30
Jul 26 15:39:34.551331 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.551306    2560 generic.go:356] "Generic (PLEG): container finished" podID="4e3d92b7-6b62-405f-af37-0064c7feb529" containerID="f13ecfb0a6e1a66c4507a5e7a88fc50b9cf58369dcd6ad33b6a02aee135a4656" exitCode=0
Jul 26 15:39:34.551441 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.551358    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-747679849d-sqgv2" event={"ID":"4e3d92b7-6b62-405f-af37-0064c7feb529","Type":"ContainerDied","Data":"f13ecfb0a6e1a66c4507a5e7a88fc50b9cf58369dcd6ad33b6a02aee135a4656"}
Jul 26 15:39:34.673052 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.673031    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:39:34.827323 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.827269    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-certificates\") pod \"4e3d92b7-6b62-405f-af37-0064c7feb529\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") "
Jul 26 15:39:34.827323 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.827301    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-installation-pull-secrets\") pod \"4e3d92b7-6b62-405f-af37-0064c7feb529\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") "
Jul 26 15:39:34.827323 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.827322    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmqk6\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-kube-api-access-gmqk6\") pod \"4e3d92b7-6b62-405f-af37-0064c7feb529\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") "
Jul 26 15:39:34.827553 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.827348    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-image-registry-private-configuration\") pod \"4e3d92b7-6b62-405f-af37-0064c7feb529\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") "
Jul 26 15:39:34.827553 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.827414    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-bound-sa-token\") pod \"4e3d92b7-6b62-405f-af37-0064c7feb529\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") "
Jul 26 15:39:34.827553 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.827465    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e3d92b7-6b62-405f-af37-0064c7feb529-ca-trust-extracted\") pod \"4e3d92b7-6b62-405f-af37-0064c7feb529\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") "
Jul 26 15:39:34.827553 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.827525    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-trusted-ca\") pod \"4e3d92b7-6b62-405f-af37-0064c7feb529\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") "
Jul 26 15:39:34.827553 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.827550    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls\") pod \"4e3d92b7-6b62-405f-af37-0064c7feb529\" (UID: \"4e3d92b7-6b62-405f-af37-0064c7feb529\") "
Jul 26 15:39:34.827811 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.827719    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4e3d92b7-6b62-405f-af37-0064c7feb529" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:39:34.828278 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.828226    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4e3d92b7-6b62-405f-af37-0064c7feb529" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:39:34.830004 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.829955    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-kube-api-access-gmqk6" (OuterVolumeSpecName: "kube-api-access-gmqk6") pod "4e3d92b7-6b62-405f-af37-0064c7feb529" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529"). InnerVolumeSpecName "kube-api-access-gmqk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:39:34.830099 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.830011    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4e3d92b7-6b62-405f-af37-0064c7feb529" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:39:34.830166 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.830097    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4e3d92b7-6b62-405f-af37-0064c7feb529" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:39:34.830166 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.830159    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4e3d92b7-6b62-405f-af37-0064c7feb529" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:39:34.830306 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.830252    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4e3d92b7-6b62-405f-af37-0064c7feb529" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:39:34.836418 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.836391    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3d92b7-6b62-405f-af37-0064c7feb529-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4e3d92b7-6b62-405f-af37-0064c7feb529" (UID: "4e3d92b7-6b62-405f-af37-0064c7feb529"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 15:39:34.928997 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.928975    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-installation-pull-secrets\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:39:34.928997 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.928994    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmqk6\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-kube-api-access-gmqk6\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:39:34.929110 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.929004    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e3d92b7-6b62-405f-af37-0064c7feb529-image-registry-private-configuration\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:39:34.929110 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.929013    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-bound-sa-token\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:39:34.929110 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.929023    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e3d92b7-6b62-405f-af37-0064c7feb529-ca-trust-extracted\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:39:34.929110 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.929031    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-trusted-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:39:34.929110 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.929039    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:39:34.929110 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:34.929047    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e3d92b7-6b62-405f-af37-0064c7feb529-registry-certificates\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:39:35.165630 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:39:35.165605    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e3d92b7_6b62_405f_af37_0064c7feb529.slice/crio-8dc78dfa0f9fc2241a7b1c9eccde78657dc801f6c6405b0a43c54f23cb3249d4\": RecentStats: unable to find data in memory cache]"
Jul 26 15:39:35.556413 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:35.556337    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-747679849d-sqgv2" event={"ID":"4e3d92b7-6b62-405f-af37-0064c7feb529","Type":"ContainerDied","Data":"8dc78dfa0f9fc2241a7b1c9eccde78657dc801f6c6405b0a43c54f23cb3249d4"}
Jul 26 15:39:35.556413 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:35.556382    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-747679849d-sqgv2"
Jul 26 15:39:35.556413 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:35.556392    2560 scope.go:117] "RemoveContainer" containerID="f13ecfb0a6e1a66c4507a5e7a88fc50b9cf58369dcd6ad33b6a02aee135a4656"
Jul 26 15:39:35.576463 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:35.576444    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-747679849d-sqgv2"]
Jul 26 15:39:35.579468 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:35.579449    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-747679849d-sqgv2"]
Jul 26 15:39:37.137489 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:37.137455    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e3d92b7-6b62-405f-af37-0064c7feb529" path="/var/lib/kubelet/pods/4e3d92b7-6b62-405f-af37-0064c7feb529/volumes"
Jul 26 15:39:50.495695 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:39:50.495665    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lkfjb"
Jul 26 15:42:25.497880 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.497846    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"]
Jul 26 15:42:25.498357 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.498079    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3d92b7-6b62-405f-af37-0064c7feb529" containerName="registry"
Jul 26 15:42:25.498357 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.498090    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3d92b7-6b62-405f-af37-0064c7feb529" containerName="registry"
Jul 26 15:42:25.498357 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.498141    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3d92b7-6b62-405f-af37-0064c7feb529" containerName="registry"
Jul 26 15:42:25.500763 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.500747    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"
Jul 26 15:42:25.503122 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.503102    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 26 15:42:25.503323 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.503309    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-mfr68\""
Jul 26 15:42:25.504050 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.504019    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 26 15:42:25.508870 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.508850    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"]
Jul 26 15:42:25.525969 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.525942    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv2sm\" (UniqueName: \"kubernetes.io/projected/37912504-017f-4ea6-a2df-4b866fb41a29-kube-api-access-jv2sm\") pod \"openshift-lws-operator-fd8ccff4c-vzrtn\" (UID: \"37912504-017f-4ea6-a2df-4b866fb41a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"
Jul 26 15:42:25.526074 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.526000    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37912504-017f-4ea6-a2df-4b866fb41a29-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vzrtn\" (UID: \"37912504-017f-4ea6-a2df-4b866fb41a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"
Jul 26 15:42:25.627269 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.627237    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37912504-017f-4ea6-a2df-4b866fb41a29-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vzrtn\" (UID: \"37912504-017f-4ea6-a2df-4b866fb41a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"
Jul 26 15:42:25.627423 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.627287    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv2sm\" (UniqueName: \"kubernetes.io/projected/37912504-017f-4ea6-a2df-4b866fb41a29-kube-api-access-jv2sm\") pod \"openshift-lws-operator-fd8ccff4c-vzrtn\" (UID: \"37912504-017f-4ea6-a2df-4b866fb41a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"
Jul 26 15:42:25.627704 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.627684    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37912504-017f-4ea6-a2df-4b866fb41a29-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vzrtn\" (UID: \"37912504-017f-4ea6-a2df-4b866fb41a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"
Jul 26 15:42:25.636063 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.636037    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv2sm\" (UniqueName: \"kubernetes.io/projected/37912504-017f-4ea6-a2df-4b866fb41a29-kube-api-access-jv2sm\") pod \"openshift-lws-operator-fd8ccff4c-vzrtn\" (UID: \"37912504-017f-4ea6-a2df-4b866fb41a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"
Jul 26 15:42:25.809954 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.809857    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"
Jul 26 15:42:25.929704 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:25.929675    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn"]
Jul 26 15:42:25.932827 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:42:25.932802    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37912504_017f_4ea6_a2df_4b866fb41a29.slice/crio-7c6977966bda4da0d125fe980592821cdf9a06e4dde3a5ea1b22529d90af2a04 WatchSource:0}: Error finding container 7c6977966bda4da0d125fe980592821cdf9a06e4dde3a5ea1b22529d90af2a04: Status 404 returned error can't find the container with id 7c6977966bda4da0d125fe980592821cdf9a06e4dde3a5ea1b22529d90af2a04
Jul 26 15:42:26.088280 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:26.088223    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn" event={"ID":"37912504-017f-4ea6-a2df-4b866fb41a29","Type":"ContainerStarted","Data":"7c6977966bda4da0d125fe980592821cdf9a06e4dde3a5ea1b22529d90af2a04"}
Jul 26 15:42:29.100945 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:29.100872    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn" event={"ID":"37912504-017f-4ea6-a2df-4b866fb41a29","Type":"ContainerStarted","Data":"5c69e8941b8ff54626ab3bdef28be4f9cf9bdf42f27bdcd4b8c09182edde6747"}
Jul 26 15:42:29.116029 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:29.115982    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vzrtn" podStartSLOduration=1.4607246649999999 podStartE2EDuration="4.115958289s" totalImagesPullingTime="2.655233624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:42:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:42:29.115015486 +0000 UTC m=+264.614405324" watchObservedRunningTime="2026-07-26 15:42:29.115958289 +0000 UTC m=+264.615348126"
Jul 26 15:42:44.381869 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.381832    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"]
Jul 26 15:42:44.385253 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.385229    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.387541 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.387520    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 26 15:42:44.387661 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.387520    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tgn28\""
Jul 26 15:42:44.387661 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.387521    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 26 15:42:44.387813 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.387799    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 26 15:42:44.387881 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.387823    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 26 15:42:44.406010 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.405969    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"]
Jul 26 15:42:44.459025 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.459006    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqlp7\" (UniqueName: \"kubernetes.io/projected/a3d0b25c-4d79-4def-99fc-8530a60fc98a-kube-api-access-kqlp7\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.459131 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.459044    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b25c-4d79-4def-99fc-8530a60fc98a-manifests\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.459131 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.459069    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b25c-4d79-4def-99fc-8530a60fc98a-tmp\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.459247 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.459168    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a3d0b25c-4d79-4def-99fc-8530a60fc98a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.459247 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.459203    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a3d0b25c-4d79-4def-99fc-8530a60fc98a-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.560310 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.560282    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a3d0b25c-4d79-4def-99fc-8530a60fc98a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.560414 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.560315    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a3d0b25c-4d79-4def-99fc-8530a60fc98a-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.560414 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.560347    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqlp7\" (UniqueName: \"kubernetes.io/projected/a3d0b25c-4d79-4def-99fc-8530a60fc98a-kube-api-access-kqlp7\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.560414 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.560395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b25c-4d79-4def-99fc-8530a60fc98a-manifests\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.560784 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.560433    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b25c-4d79-4def-99fc-8530a60fc98a-tmp\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.560834 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.560821    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b25c-4d79-4def-99fc-8530a60fc98a-manifests\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.562613 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.562593    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3d0b25c-4d79-4def-99fc-8530a60fc98a-tmp\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.562695 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.562599    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a3d0b25c-4d79-4def-99fc-8530a60fc98a-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.562825 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.562805    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a3d0b25c-4d79-4def-99fc-8530a60fc98a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.567858 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.567836    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqlp7\" (UniqueName: \"kubernetes.io/projected/a3d0b25c-4d79-4def-99fc-8530a60fc98a-kube-api-access-kqlp7\") pod \"opendatahub-operator-controller-manager-66d4685587-6w2v4\" (UID: \"a3d0b25c-4d79-4def-99fc-8530a60fc98a\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.694991 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.694963    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:44.822290 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:44.822257    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"]
Jul 26 15:42:44.826402 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:42:44.826372    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d0b25c_4d79_4def_99fc_8530a60fc98a.slice/crio-472d54cbea9df852f45bdf963952200ffd63deff8e82bf8001b8b9c1cfaf59c1 WatchSource:0}: Error finding container 472d54cbea9df852f45bdf963952200ffd63deff8e82bf8001b8b9c1cfaf59c1: Status 404 returned error can't find the container with id 472d54cbea9df852f45bdf963952200ffd63deff8e82bf8001b8b9c1cfaf59c1
Jul 26 15:42:45.153587 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:45.153555    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4" event={"ID":"a3d0b25c-4d79-4def-99fc-8530a60fc98a","Type":"ContainerStarted","Data":"472d54cbea9df852f45bdf963952200ffd63deff8e82bf8001b8b9c1cfaf59c1"}
Jul 26 15:42:48.165693 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:48.165668    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4" event={"ID":"a3d0b25c-4d79-4def-99fc-8530a60fc98a","Type":"ContainerStarted","Data":"f6df4e95f170e4b82a6034e6f893d5ef0fa82cb021f4c77544b094ffceba0645"}
Jul 26 15:42:49.170291 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:49.170255    2560 generic.go:356] "Generic (PLEG): container finished" podID="a3d0b25c-4d79-4def-99fc-8530a60fc98a" containerID="f6df4e95f170e4b82a6034e6f893d5ef0fa82cb021f4c77544b094ffceba0645" exitCode=0
Jul 26 15:42:49.170624 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:49.170316    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4" event={"ID":"a3d0b25c-4d79-4def-99fc-8530a60fc98a","Type":"ContainerDied","Data":"f6df4e95f170e4b82a6034e6f893d5ef0fa82cb021f4c77544b094ffceba0645"}
Jul 26 15:42:50.175057 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:50.175021    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4" event={"ID":"a3d0b25c-4d79-4def-99fc-8530a60fc98a","Type":"ContainerStarted","Data":"912221a733f3fbe513b3462d03cddce6e77c2e024b4b227f7a5dc735420edb50"}
Jul 26 15:42:50.175440 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:50.175219    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:42:50.195809 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:42:50.195760    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4" podStartSLOduration=1.9437383860000002 podStartE2EDuration="6.195746252s" totalImagesPullingTime="3.258398948s" totalInitContainerRuntime="993.608918ms" isStatefulPod=false podCreationTimestamp="2026-07-26 15:42:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:42:50.194667897 +0000 UTC m=+285.694057746" watchObservedRunningTime="2026-07-26 15:42:50.195746252 +0000 UTC m=+285.695136090"
Jul 26 15:43:01.179887 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:01.179855    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-6w2v4"
Jul 26 15:43:04.910420 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:04.910391    2560 kubelet.go:1648] "Image garbage collection succeeded"
Jul 26 15:43:13.124876 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.124843    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"]
Jul 26 15:43:13.126931 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.126885    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"
Jul 26 15:43:13.130201 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.130181    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 26 15:43:13.130460 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.130439    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-756j7\""
Jul 26 15:43:13.143335 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.143310    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"]
Jul 26 15:43:13.156522 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.156497    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"]
Jul 26 15:43:13.158974 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.158958    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"
Jul 26 15:43:13.161240 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.161222    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 26 15:43:13.161733 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.161716    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wcjsj\""
Jul 26 15:43:13.178091 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.178071    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"]
Jul 26 15:43:13.264509 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.264228    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fxvt\" (UniqueName: \"kubernetes.io/projected/a1ef07ad-8d81-4dc9-9f94-160eef929d4f-kube-api-access-5fxvt\") pod \"llmisvc-controller-manager-777c6bd7d5-sgtxq\" (UID: \"a1ef07ad-8d81-4dc9-9f94-160eef929d4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"
Jul 26 15:43:13.264509 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.264329    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf2nv\" (UniqueName: \"kubernetes.io/projected/49024e42-a310-4da5-b905-cb44535548a0-kube-api-access-qf2nv\") pod \"kserve-controller-manager-d6f4b65b8-qmwhg\" (UID: \"49024e42-a310-4da5-b905-cb44535548a0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"
Jul 26 15:43:13.264509 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.264362    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1ef07ad-8d81-4dc9-9f94-160eef929d4f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sgtxq\" (UID: \"a1ef07ad-8d81-4dc9-9f94-160eef929d4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"
Jul 26 15:43:13.264509 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.264415    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49024e42-a310-4da5-b905-cb44535548a0-cert\") pod \"kserve-controller-manager-d6f4b65b8-qmwhg\" (UID: \"49024e42-a310-4da5-b905-cb44535548a0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"
Jul 26 15:43:13.365584 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.365546    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49024e42-a310-4da5-b905-cb44535548a0-cert\") pod \"kserve-controller-manager-d6f4b65b8-qmwhg\" (UID: \"49024e42-a310-4da5-b905-cb44535548a0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"
Jul 26 15:43:13.365748 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.365587    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fxvt\" (UniqueName: \"kubernetes.io/projected/a1ef07ad-8d81-4dc9-9f94-160eef929d4f-kube-api-access-5fxvt\") pod \"llmisvc-controller-manager-777c6bd7d5-sgtxq\" (UID: \"a1ef07ad-8d81-4dc9-9f94-160eef929d4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"
Jul 26 15:43:13.365748 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.365659    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf2nv\" (UniqueName: \"kubernetes.io/projected/49024e42-a310-4da5-b905-cb44535548a0-kube-api-access-qf2nv\") pod \"kserve-controller-manager-d6f4b65b8-qmwhg\" (UID: \"49024e42-a310-4da5-b905-cb44535548a0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"
Jul 26 15:43:13.365748 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.365684    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1ef07ad-8d81-4dc9-9f94-160eef929d4f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sgtxq\" (UID: \"a1ef07ad-8d81-4dc9-9f94-160eef929d4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"
Jul 26 15:43:13.367969 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.367943    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49024e42-a310-4da5-b905-cb44535548a0-cert\") pod \"kserve-controller-manager-d6f4b65b8-qmwhg\" (UID: \"49024e42-a310-4da5-b905-cb44535548a0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"
Jul 26 15:43:13.368075 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.368017    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1ef07ad-8d81-4dc9-9f94-160eef929d4f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sgtxq\" (UID: \"a1ef07ad-8d81-4dc9-9f94-160eef929d4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"
Jul 26 15:43:13.384727 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.384672    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fxvt\" (UniqueName: \"kubernetes.io/projected/a1ef07ad-8d81-4dc9-9f94-160eef929d4f-kube-api-access-5fxvt\") pod \"llmisvc-controller-manager-777c6bd7d5-sgtxq\" (UID: \"a1ef07ad-8d81-4dc9-9f94-160eef929d4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"
Jul 26 15:43:13.399208 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.399191    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf2nv\" (UniqueName: \"kubernetes.io/projected/49024e42-a310-4da5-b905-cb44535548a0-kube-api-access-qf2nv\") pod \"kserve-controller-manager-d6f4b65b8-qmwhg\" (UID: \"49024e42-a310-4da5-b905-cb44535548a0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"
Jul 26 15:43:13.437068 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.437043    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"
Jul 26 15:43:13.468006 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.467984    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"
Jul 26 15:43:13.599870 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.599839    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"]
Jul 26 15:43:13.604146 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:43:13.604116    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49024e42_a310_4da5_b905_cb44535548a0.slice/crio-f237f887e8362747fa3e63ae11613fed15d14e8b7f5d80024165b1ab557a5074 WatchSource:0}: Error finding container f237f887e8362747fa3e63ae11613fed15d14e8b7f5d80024165b1ab557a5074: Status 404 returned error can't find the container with id f237f887e8362747fa3e63ae11613fed15d14e8b7f5d80024165b1ab557a5074
Jul 26 15:43:13.605398 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.605381    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:43:13.614135 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:13.614118    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"]
Jul 26 15:43:13.616128 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:43:13.616103    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda1ef07ad_8d81_4dc9_9f94_160eef929d4f.slice/crio-4070aa9fcd83d3012818f2e22dfbef034653d9909aa32a46bfdedec86c11f4d9 WatchSource:0}: Error finding container 4070aa9fcd83d3012818f2e22dfbef034653d9909aa32a46bfdedec86c11f4d9: Status 404 returned error can't find the container with id 4070aa9fcd83d3012818f2e22dfbef034653d9909aa32a46bfdedec86c11f4d9
Jul 26 15:43:14.039035 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.039004    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"]
Jul 26 15:43:14.041807 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.041788    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.045639 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.045621    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 26 15:43:14.046172 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.046154    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 26 15:43:14.046265 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.046159    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-n48rr\""
Jul 26 15:43:14.046473 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.046454    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 26 15:43:14.047844 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.047827    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 26 15:43:14.070624 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.070605    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"]
Jul 26 15:43:14.172786 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.172759    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fccad28-d8c4-4e35-b2ee-31aff78e5df3-tmp\") pod \"kube-auth-proxy-694bf5b6d6-f64vz\" (UID: \"3fccad28-d8c4-4e35-b2ee-31aff78e5df3\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.173238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.172794    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjts4\" (UniqueName: \"kubernetes.io/projected/3fccad28-d8c4-4e35-b2ee-31aff78e5df3-kube-api-access-wjts4\") pod \"kube-auth-proxy-694bf5b6d6-f64vz\" (UID: \"3fccad28-d8c4-4e35-b2ee-31aff78e5df3\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.173238 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.172819    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fccad28-d8c4-4e35-b2ee-31aff78e5df3-tls-certs\") pod \"kube-auth-proxy-694bf5b6d6-f64vz\" (UID: \"3fccad28-d8c4-4e35-b2ee-31aff78e5df3\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.258993 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.258923    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq" event={"ID":"a1ef07ad-8d81-4dc9-9f94-160eef929d4f","Type":"ContainerStarted","Data":"4070aa9fcd83d3012818f2e22dfbef034653d9909aa32a46bfdedec86c11f4d9"}
Jul 26 15:43:14.260619 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.260582    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg" event={"ID":"49024e42-a310-4da5-b905-cb44535548a0","Type":"ContainerStarted","Data":"f237f887e8362747fa3e63ae11613fed15d14e8b7f5d80024165b1ab557a5074"}
Jul 26 15:43:14.274252 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.274216    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjts4\" (UniqueName: \"kubernetes.io/projected/3fccad28-d8c4-4e35-b2ee-31aff78e5df3-kube-api-access-wjts4\") pod \"kube-auth-proxy-694bf5b6d6-f64vz\" (UID: \"3fccad28-d8c4-4e35-b2ee-31aff78e5df3\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.274378 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.274264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fccad28-d8c4-4e35-b2ee-31aff78e5df3-tls-certs\") pod \"kube-auth-proxy-694bf5b6d6-f64vz\" (UID: \"3fccad28-d8c4-4e35-b2ee-31aff78e5df3\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.274378 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.274364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fccad28-d8c4-4e35-b2ee-31aff78e5df3-tmp\") pod \"kube-auth-proxy-694bf5b6d6-f64vz\" (UID: \"3fccad28-d8c4-4e35-b2ee-31aff78e5df3\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.276993 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.276962    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fccad28-d8c4-4e35-b2ee-31aff78e5df3-tmp\") pod \"kube-auth-proxy-694bf5b6d6-f64vz\" (UID: \"3fccad28-d8c4-4e35-b2ee-31aff78e5df3\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.277183 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.277163    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fccad28-d8c4-4e35-b2ee-31aff78e5df3-tls-certs\") pod \"kube-auth-proxy-694bf5b6d6-f64vz\" (UID: \"3fccad28-d8c4-4e35-b2ee-31aff78e5df3\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.285138 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.285095    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjts4\" (UniqueName: \"kubernetes.io/projected/3fccad28-d8c4-4e35-b2ee-31aff78e5df3-kube-api-access-wjts4\") pod \"kube-auth-proxy-694bf5b6d6-f64vz\" (UID: \"3fccad28-d8c4-4e35-b2ee-31aff78e5df3\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.351730 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.351664    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"
Jul 26 15:43:14.734656 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:14.734621    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz"]
Jul 26 15:43:14.856828 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:43:14.856791    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fccad28_d8c4_4e35_b2ee_31aff78e5df3.slice/crio-08b55f78afcc73115842798032bf1584e89df96792a81d24ee9c8f54ad4d345c WatchSource:0}: Error finding container 08b55f78afcc73115842798032bf1584e89df96792a81d24ee9c8f54ad4d345c: Status 404 returned error can't find the container with id 08b55f78afcc73115842798032bf1584e89df96792a81d24ee9c8f54ad4d345c
Jul 26 15:43:15.267171 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:15.267137    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz" event={"ID":"3fccad28-d8c4-4e35-b2ee-31aff78e5df3","Type":"ContainerStarted","Data":"08b55f78afcc73115842798032bf1584e89df96792a81d24ee9c8f54ad4d345c"}
Jul 26 15:43:16.154859 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.152472    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"]
Jul 26 15:43:16.156587 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.156557    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.160019 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.159994    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 26 15:43:16.160706 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.160684    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 26 15:43:16.160816 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.160723    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m4nmg\""
Jul 26 15:43:16.161009 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.160987    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 26 15:43:16.161141 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.161120    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 26 15:43:16.173600 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.173578    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"]
Jul 26 15:43:16.274199 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.274154    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg" event={"ID":"49024e42-a310-4da5-b905-cb44535548a0","Type":"ContainerStarted","Data":"5bec5420f97348751d3f738d25ea00c0ebcbf57ef5f89d779f2a2da1125e30fb"}
Jul 26 15:43:16.274630 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.274312    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"
Jul 26 15:43:16.291558 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.291530    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fdc748d3-eddb-4153-b18a-247da49f480f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.291690 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.291576    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fdc748d3-eddb-4153-b18a-247da49f480f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.291690 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.291606    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fdc748d3-eddb-4153-b18a-247da49f480f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.291690 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.291672    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fdc748d3-eddb-4153-b18a-247da49f480f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.291852 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.291701    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpr6q\" (UniqueName: \"kubernetes.io/projected/fdc748d3-eddb-4153-b18a-247da49f480f-kube-api-access-zpr6q\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.291852 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.291725    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdc748d3-eddb-4153-b18a-247da49f480f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.291852 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.291745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdc748d3-eddb-4153-b18a-247da49f480f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.297792 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.297728    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg" podStartSLOduration=1.077055068 podStartE2EDuration="3.297709805s" totalImagesPullingTime="2.220654737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:43:16.296014273 +0000 UTC m=+311.795404113" watchObservedRunningTime="2026-07-26 15:43:16.297709805 +0000 UTC m=+311.797099645"
Jul 26 15:43:16.392494 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.392453    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fdc748d3-eddb-4153-b18a-247da49f480f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.392494 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.392494    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpr6q\" (UniqueName: \"kubernetes.io/projected/fdc748d3-eddb-4153-b18a-247da49f480f-kube-api-access-zpr6q\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.392705 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.392523    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdc748d3-eddb-4153-b18a-247da49f480f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.392705 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.392559    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdc748d3-eddb-4153-b18a-247da49f480f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.392705 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.392643    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fdc748d3-eddb-4153-b18a-247da49f480f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.392705 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.392683    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fdc748d3-eddb-4153-b18a-247da49f480f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.393001 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.392708    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fdc748d3-eddb-4153-b18a-247da49f480f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.393352 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.393324    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fdc748d3-eddb-4153-b18a-247da49f480f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.395604 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.395578    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdc748d3-eddb-4153-b18a-247da49f480f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.395702 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.395672    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fdc748d3-eddb-4153-b18a-247da49f480f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.395872 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.395849    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fdc748d3-eddb-4153-b18a-247da49f480f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.396152 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.396132    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fdc748d3-eddb-4153-b18a-247da49f480f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.408513 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.408459    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdc748d3-eddb-4153-b18a-247da49f480f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.408991 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.408966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpr6q\" (UniqueName: \"kubernetes.io/projected/fdc748d3-eddb-4153-b18a-247da49f480f-kube-api-access-zpr6q\") pod \"istiod-openshift-gateway-5677ccb49b-klmb9\" (UID: \"fdc748d3-eddb-4153-b18a-247da49f480f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:16.467797 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:16.467768    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:17.721744 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:17.721648    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"]
Jul 26 15:43:17.723937 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:43:17.723908    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc748d3_eddb_4153_b18a_247da49f480f.slice/crio-29cd4c131349b3536358cbd380b366f598044c8104b4b1d222e34fe3d85db24a WatchSource:0}: Error finding container 29cd4c131349b3536358cbd380b366f598044c8104b4b1d222e34fe3d85db24a: Status 404 returned error can't find the container with id 29cd4c131349b3536358cbd380b366f598044c8104b4b1d222e34fe3d85db24a
Jul 26 15:43:18.284120 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:18.284081    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9" event={"ID":"fdc748d3-eddb-4153-b18a-247da49f480f","Type":"ContainerStarted","Data":"29cd4c131349b3536358cbd380b366f598044c8104b4b1d222e34fe3d85db24a"}
Jul 26 15:43:18.285413 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:18.285385    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz" event={"ID":"3fccad28-d8c4-4e35-b2ee-31aff78e5df3","Type":"ContainerStarted","Data":"33ce2ad69d2716afff369d72130f418acf5f5197c48ae76d24127a8106567bc4"}
Jul 26 15:43:18.286786 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:18.286764    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq" event={"ID":"a1ef07ad-8d81-4dc9-9f94-160eef929d4f","Type":"ContainerStarted","Data":"689183e66eb6a659e6e3f0ff4794bb42c3176c86c82aa8bf5285d2f865b69740"}
Jul 26 15:43:18.286877 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:18.286829    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"
Jul 26 15:43:18.304000 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:18.303962    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-f64vz" podStartSLOduration=1.562464055 podStartE2EDuration="4.303947989s" totalImagesPullingTime="2.741483934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:43:18.302925433 +0000 UTC m=+313.802315263" watchObservedRunningTime="2026-07-26 15:43:18.303947989 +0000 UTC m=+313.803337806"
Jul 26 15:43:18.322978 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:18.322929    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq" podStartSLOduration=1.342559063 podStartE2EDuration="5.322885432s" totalImagesPullingTime="3.980326369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:43:18.321356911 +0000 UTC m=+313.820746748" watchObservedRunningTime="2026-07-26 15:43:18.322885432 +0000 UTC m=+313.822275270"
Jul 26 15:43:20.342108 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:20.342064    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 15:43:20.342343 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:20.342153    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 15:43:21.302443 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:21.302410    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9" event={"ID":"fdc748d3-eddb-4153-b18a-247da49f480f","Type":"ContainerStarted","Data":"2b052d45df669be11d33a416d6928199418d9753b61edcda07dd0e7763f3a2ad"}
Jul 26 15:43:21.302629 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:21.302606    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:21.304223 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:21.304196    2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-klmb9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 26 15:43:21.304314 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:21.304252    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9" podUID="fdc748d3-eddb-4153-b18a-247da49f480f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 15:43:21.322032 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:21.321978    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9" podStartSLOduration=2.706556532 podStartE2EDuration="5.321962283s" totalImagesPullingTime="2.615405751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:43:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:43:21.320978245 +0000 UTC m=+316.820368082" watchObservedRunningTime="2026-07-26 15:43:21.321962283 +0000 UTC m=+316.821352122"
Jul 26 15:43:22.306929 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:22.306875    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-klmb9"
Jul 26 15:43:47.284033 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:47.283998    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmwhg"
Jul 26 15:43:49.293089 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:43:49.293063    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sgtxq"
Jul 26 15:44:40.444516 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.444482    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"]
Jul 26 15:44:40.446795 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.446774    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:44:40.449316 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.449291    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 26 15:44:40.449423 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.449317    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 26 15:44:40.450291 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.450272    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-wct5p\""
Jul 26 15:44:40.458310 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.458244    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"]
Jul 26 15:44:40.605022 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.604990    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lk9q\" (UniqueName: \"kubernetes.io/projected/6140299b-d1e1-4a5a-a792-34941e400919-kube-api-access-9lk9q\") pod \"limitador-operator-controller-manager-859d7b78bd-lcmlk\" (UID: \"6140299b-d1e1-4a5a-a792-34941e400919\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:44:40.705940 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.705833    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lk9q\" (UniqueName: \"kubernetes.io/projected/6140299b-d1e1-4a5a-a792-34941e400919-kube-api-access-9lk9q\") pod \"limitador-operator-controller-manager-859d7b78bd-lcmlk\" (UID: \"6140299b-d1e1-4a5a-a792-34941e400919\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:44:40.715121 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.715087    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lk9q\" (UniqueName: \"kubernetes.io/projected/6140299b-d1e1-4a5a-a792-34941e400919-kube-api-access-9lk9q\") pod \"limitador-operator-controller-manager-859d7b78bd-lcmlk\" (UID: \"6140299b-d1e1-4a5a-a792-34941e400919\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:44:40.756812 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.756785    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:44:40.889068 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:40.889035    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"]
Jul 26 15:44:40.891845 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:44:40.891814    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6140299b_d1e1_4a5a_a792_34941e400919.slice/crio-bfd44aabeaabddc70c3a22d2c56a5372acd43d6b4c6326b46d20f4eb20b35a16 WatchSource:0}: Error finding container bfd44aabeaabddc70c3a22d2c56a5372acd43d6b4c6326b46d20f4eb20b35a16: Status 404 returned error can't find the container with id bfd44aabeaabddc70c3a22d2c56a5372acd43d6b4c6326b46d20f4eb20b35a16
Jul 26 15:44:41.603204 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:41.603166    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk" event={"ID":"6140299b-d1e1-4a5a-a792-34941e400919","Type":"ContainerStarted","Data":"bfd44aabeaabddc70c3a22d2c56a5372acd43d6b4c6326b46d20f4eb20b35a16"}
Jul 26 15:44:43.613155 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:43.613117    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk" event={"ID":"6140299b-d1e1-4a5a-a792-34941e400919","Type":"ContainerStarted","Data":"4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28"}
Jul 26 15:44:43.613508 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:43.613245    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:44:43.649399 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:43.649356    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk" podStartSLOduration=1.949594935 podStartE2EDuration="3.649342262s" totalImagesPullingTime="1.699747327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:44:43.645507209 +0000 UTC m=+399.144897047" watchObservedRunningTime="2026-07-26 15:44:43.649342262 +0000 UTC m=+399.148732167"
Jul 26 15:44:54.619937 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:44:54.619876    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:45:06.712916 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.711947    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"]
Jul 26 15:45:06.712916 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.712182    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk" podUID="6140299b-d1e1-4a5a-a792-34941e400919" containerName="manager" containerID="cri-o://4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28" gracePeriod=2
Jul 26 15:45:06.713523 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.713044    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"]
Jul 26 15:45:06.716806 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:45:06.716772    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lcmlk\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="6140299b-d1e1-4a5a-a792-34941e400919" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:45:06.729977 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.729952    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h"]
Jul 26 15:45:06.730341 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.730325    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6140299b-d1e1-4a5a-a792-34941e400919" containerName="manager"
Jul 26 15:45:06.730383 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.730346    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6140299b-d1e1-4a5a-a792-34941e400919" containerName="manager"
Jul 26 15:45:06.730424 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.730413    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6140299b-d1e1-4a5a-a792-34941e400919" containerName="manager"
Jul 26 15:45:06.733445 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.733421    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h"
Jul 26 15:45:06.735440 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:45:06.735405    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lcmlk\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="6140299b-d1e1-4a5a-a792-34941e400919" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:45:06.742757 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.742733    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h"]
Jul 26 15:45:06.880282 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.880253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj2sw\" (UniqueName: \"kubernetes.io/projected/dfe51eeb-0d36-455a-a8e7-63e1d4ec1f99-kube-api-access-zj2sw\") pod \"limitador-operator-controller-manager-859d7b78bd-cgq5h\" (UID: \"dfe51eeb-0d36-455a-a8e7-63e1d4ec1f99\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h"
Jul 26 15:45:06.942146 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.942123    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:45:06.944178 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:45:06.944152    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lcmlk\" is forbidden: User \"system:node:ip-10-0-132-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-10.ec2.internal' and this object" podUID="6140299b-d1e1-4a5a-a792-34941e400919" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:45:06.981645 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.981590    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj2sw\" (UniqueName: \"kubernetes.io/projected/dfe51eeb-0d36-455a-a8e7-63e1d4ec1f99-kube-api-access-zj2sw\") pod \"limitador-operator-controller-manager-859d7b78bd-cgq5h\" (UID: \"dfe51eeb-0d36-455a-a8e7-63e1d4ec1f99\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h"
Jul 26 15:45:06.998560 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:06.998533    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj2sw\" (UniqueName: \"kubernetes.io/projected/dfe51eeb-0d36-455a-a8e7-63e1d4ec1f99-kube-api-access-zj2sw\") pod \"limitador-operator-controller-manager-859d7b78bd-cgq5h\" (UID: \"dfe51eeb-0d36-455a-a8e7-63e1d4ec1f99\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h"
Jul 26 15:45:07.082570 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.082539    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lk9q\" (UniqueName: \"kubernetes.io/projected/6140299b-d1e1-4a5a-a792-34941e400919-kube-api-access-9lk9q\") pod \"6140299b-d1e1-4a5a-a792-34941e400919\" (UID: \"6140299b-d1e1-4a5a-a792-34941e400919\") "
Jul 26 15:45:07.084455 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.084431    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6140299b-d1e1-4a5a-a792-34941e400919-kube-api-access-9lk9q" (OuterVolumeSpecName: "kube-api-access-9lk9q") pod "6140299b-d1e1-4a5a-a792-34941e400919" (UID: "6140299b-d1e1-4a5a-a792-34941e400919"). InnerVolumeSpecName "kube-api-access-9lk9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:45:07.098566 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.098550    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h"
Jul 26 15:45:07.138879 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.138854    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6140299b-d1e1-4a5a-a792-34941e400919" path="/var/lib/kubelet/pods/6140299b-d1e1-4a5a-a792-34941e400919/volumes"
Jul 26 15:45:07.183629 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.183607    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lk9q\" (UniqueName: \"kubernetes.io/projected/6140299b-d1e1-4a5a-a792-34941e400919-kube-api-access-9lk9q\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:45:07.222082 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.222047    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h"]
Jul 26 15:45:07.225545 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:45:07.225518    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfe51eeb_0d36_455a_a8e7_63e1d4ec1f99.slice/crio-a76e08a04a88ca63a5bf3515041063dedd712e8a66934d5a6685414a67e8cc3e WatchSource:0}: Error finding container a76e08a04a88ca63a5bf3515041063dedd712e8a66934d5a6685414a67e8cc3e: Status 404 returned error can't find the container with id a76e08a04a88ca63a5bf3515041063dedd712e8a66934d5a6685414a67e8cc3e
Jul 26 15:45:07.723490 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.723451    2560 generic.go:356] "Generic (PLEG): container finished" podID="6140299b-d1e1-4a5a-a792-34941e400919" containerID="4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28" exitCode=0
Jul 26 15:45:07.723949 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.723509    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lcmlk"
Jul 26 15:45:07.723949 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.723559    2560 scope.go:117] "RemoveContainer" containerID="4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28"
Jul 26 15:45:07.725201 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.725179    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h" event={"ID":"dfe51eeb-0d36-455a-a8e7-63e1d4ec1f99","Type":"ContainerStarted","Data":"a76a244f77621f3303d20e4758389666d0fbd1108f618e92d88024e6f28f694f"}
Jul 26 15:45:07.725316 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.725261    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h" event={"ID":"dfe51eeb-0d36-455a-a8e7-63e1d4ec1f99","Type":"ContainerStarted","Data":"a76e08a04a88ca63a5bf3515041063dedd712e8a66934d5a6685414a67e8cc3e"}
Jul 26 15:45:07.725382 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.725318    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h"
Jul 26 15:45:07.738173 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.738134    2560 scope.go:117] "RemoveContainer" containerID="4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28"
Jul 26 15:45:07.738435 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:45:07.738407    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28\": container with ID starting with 4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28 not found: ID does not exist" containerID="4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28"
Jul 26 15:45:07.738481 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.738444    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28"} err="failed to get container status \"4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28\": rpc error: code = NotFound desc = could not find container \"4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28\": container with ID starting with 4272e7db4478ef8a504e5266ba2b855a7851bffb128d4274655cab32486c2c28 not found: ID does not exist"
Jul 26 15:45:07.753270 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:07.753233    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h" podStartSLOduration=1.7532216919999999 podStartE2EDuration="1.753221692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:45:07.75246211 +0000 UTC m=+423.251851948" watchObservedRunningTime="2026-07-26 15:45:07.753221692 +0000 UTC m=+423.252611591"
Jul 26 15:45:18.732616 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:18.732587    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgq5h"
Jul 26 15:45:52.872498 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.872465    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"]
Jul 26 15:45:52.875622 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.875604    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:45:52.877566 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.877537    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 26 15:45:52.877679 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.877595    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sjb8w\""
Jul 26 15:45:52.883788 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.883767    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"]
Jul 26 15:45:52.894565 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.894544    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/07401a79-9820-4c09-9e3e-27fa971f41e5-config-file\") pod \"limitador-limitador-64c94fd64f-hm2qx\" (UID: \"07401a79-9820-4c09-9e3e-27fa971f41e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:45:52.894647 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.894591    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfvbn\" (UniqueName: \"kubernetes.io/projected/07401a79-9820-4c09-9e3e-27fa971f41e5-kube-api-access-lfvbn\") pod \"limitador-limitador-64c94fd64f-hm2qx\" (UID: \"07401a79-9820-4c09-9e3e-27fa971f41e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:45:52.974372 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.974342    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"]
Jul 26 15:45:52.995539 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.995510    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/07401a79-9820-4c09-9e3e-27fa971f41e5-config-file\") pod \"limitador-limitador-64c94fd64f-hm2qx\" (UID: \"07401a79-9820-4c09-9e3e-27fa971f41e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:45:52.999299 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.995678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfvbn\" (UniqueName: \"kubernetes.io/projected/07401a79-9820-4c09-9e3e-27fa971f41e5-kube-api-access-lfvbn\") pod \"limitador-limitador-64c94fd64f-hm2qx\" (UID: \"07401a79-9820-4c09-9e3e-27fa971f41e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:45:52.999299 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:52.996304    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/07401a79-9820-4c09-9e3e-27fa971f41e5-config-file\") pod \"limitador-limitador-64c94fd64f-hm2qx\" (UID: \"07401a79-9820-4c09-9e3e-27fa971f41e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:45:53.003262 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:53.003242    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfvbn\" (UniqueName: \"kubernetes.io/projected/07401a79-9820-4c09-9e3e-27fa971f41e5-kube-api-access-lfvbn\") pod \"limitador-limitador-64c94fd64f-hm2qx\" (UID: \"07401a79-9820-4c09-9e3e-27fa971f41e5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:45:53.186507 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:53.186472    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:45:53.310022 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:53.309996    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"]
Jul 26 15:45:53.312662 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:45:53.312635    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07401a79_9820_4c09_9e3e_27fa971f41e5.slice/crio-46a9c23b818ea8e65c2f25c9ecd218350381a698cb301026fbd0f0dde737c90d WatchSource:0}: Error finding container 46a9c23b818ea8e65c2f25c9ecd218350381a698cb301026fbd0f0dde737c90d: Status 404 returned error can't find the container with id 46a9c23b818ea8e65c2f25c9ecd218350381a698cb301026fbd0f0dde737c90d
Jul 26 15:45:53.904383 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:53.904349    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx" event={"ID":"07401a79-9820-4c09-9e3e-27fa971f41e5","Type":"ContainerStarted","Data":"46a9c23b818ea8e65c2f25c9ecd218350381a698cb301026fbd0f0dde737c90d"}
Jul 26 15:45:56.917977 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:56.917944    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx" event={"ID":"07401a79-9820-4c09-9e3e-27fa971f41e5","Type":"ContainerStarted","Data":"30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715"}
Jul 26 15:45:56.918363 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:56.918098    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:45:56.933666 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:45:56.933619    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx" podStartSLOduration=2.330129986 podStartE2EDuration="4.93360326s" totalImagesPullingTime="2.603473274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:45:56.932727363 +0000 UTC m=+472.432117212" watchObservedRunningTime="2026-07-26 15:45:56.93360326 +0000 UTC m=+472.432993097"
Jul 26 15:46:07.922638 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:07.922608    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:46:09.664836 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:09.664796    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"]
Jul 26 15:46:09.665326 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:09.665027    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx" podUID="07401a79-9820-4c09-9e3e-27fa971f41e5" containerName="limitador" containerID="cri-o://30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715" gracePeriod=30
Jul 26 15:46:10.201715 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.201686    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:46:10.312698 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.312603    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/07401a79-9820-4c09-9e3e-27fa971f41e5-config-file\") pod \"07401a79-9820-4c09-9e3e-27fa971f41e5\" (UID: \"07401a79-9820-4c09-9e3e-27fa971f41e5\") "
Jul 26 15:46:10.312698 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.312666    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfvbn\" (UniqueName: \"kubernetes.io/projected/07401a79-9820-4c09-9e3e-27fa971f41e5-kube-api-access-lfvbn\") pod \"07401a79-9820-4c09-9e3e-27fa971f41e5\" (UID: \"07401a79-9820-4c09-9e3e-27fa971f41e5\") "
Jul 26 15:46:10.312962 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.312940    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07401a79-9820-4c09-9e3e-27fa971f41e5-config-file" (OuterVolumeSpecName: "config-file") pod "07401a79-9820-4c09-9e3e-27fa971f41e5" (UID: "07401a79-9820-4c09-9e3e-27fa971f41e5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:46:10.314684 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.314659    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07401a79-9820-4c09-9e3e-27fa971f41e5-kube-api-access-lfvbn" (OuterVolumeSpecName: "kube-api-access-lfvbn") pod "07401a79-9820-4c09-9e3e-27fa971f41e5" (UID: "07401a79-9820-4c09-9e3e-27fa971f41e5"). InnerVolumeSpecName "kube-api-access-lfvbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:46:10.413542 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.413495    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/07401a79-9820-4c09-9e3e-27fa971f41e5-config-file\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:46:10.413542 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.413535    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfvbn\" (UniqueName: \"kubernetes.io/projected/07401a79-9820-4c09-9e3e-27fa971f41e5-kube-api-access-lfvbn\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:46:10.974675 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.974632    2560 generic.go:356] "Generic (PLEG): container finished" podID="07401a79-9820-4c09-9e3e-27fa971f41e5" containerID="30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715" exitCode=0
Jul 26 15:46:10.975107 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.974688    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"
Jul 26 15:46:10.975107 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.974720    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx" event={"ID":"07401a79-9820-4c09-9e3e-27fa971f41e5","Type":"ContainerDied","Data":"30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715"}
Jul 26 15:46:10.975107 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.974772    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hm2qx" event={"ID":"07401a79-9820-4c09-9e3e-27fa971f41e5","Type":"ContainerDied","Data":"46a9c23b818ea8e65c2f25c9ecd218350381a698cb301026fbd0f0dde737c90d"}
Jul 26 15:46:10.975107 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.974794    2560 scope.go:117] "RemoveContainer" containerID="30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715"
Jul 26 15:46:10.982630 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.982610    2560 scope.go:117] "RemoveContainer" containerID="30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715"
Jul 26 15:46:10.982872 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:46:10.982854    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715\": container with ID starting with 30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715 not found: ID does not exist" containerID="30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715"
Jul 26 15:46:10.982975 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.982878    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715"} err="failed to get container status \"30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715\": rpc error: code = NotFound desc = could not find container \"30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715\": container with ID starting with 30fb67d600181a96e824a3754ec90442ee0e743d9ad980b61a2ccf897abbe715 not found: ID does not exist"
Jul 26 15:46:10.992611 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.992592    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"]
Jul 26 15:46:10.995373 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:10.995352    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hm2qx"]
Jul 26 15:46:11.138218 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:11.138183    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07401a79-9820-4c09-9e3e-27fa971f41e5" path="/var/lib/kubelet/pods/07401a79-9820-4c09-9e3e-27fa971f41e5/volumes"
Jul 26 15:46:13.869450 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.869417    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"]
Jul 26 15:46:13.869985 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.869692    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07401a79-9820-4c09-9e3e-27fa971f41e5" containerName="limitador"
Jul 26 15:46:13.869985 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.869703    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="07401a79-9820-4c09-9e3e-27fa971f41e5" containerName="limitador"
Jul 26 15:46:13.869985 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.869773    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="07401a79-9820-4c09-9e3e-27fa971f41e5" containerName="limitador"
Jul 26 15:46:13.874305 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.874284    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"
Jul 26 15:46:13.876205 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.876183    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 26 15:46:13.876336 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.876183    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 15:46:13.876853 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.876827    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 26 15:46:13.876950 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.876828    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-h8mg8\""
Jul 26 15:46:13.880418 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.880395    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"]
Jul 26 15:46:13.942009 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.941968    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/759acf50-d889-44a2-94c4-f20d52ec3b1f-data\") pod \"postgres-69fb4594cf-pd89t\" (UID: \"759acf50-d889-44a2-94c4-f20d52ec3b1f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"
Jul 26 15:46:13.942260 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:13.942018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt6fk\" (UniqueName: \"kubernetes.io/projected/759acf50-d889-44a2-94c4-f20d52ec3b1f-kube-api-access-kt6fk\") pod \"postgres-69fb4594cf-pd89t\" (UID: \"759acf50-d889-44a2-94c4-f20d52ec3b1f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"
Jul 26 15:46:14.042632 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:14.042599    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/759acf50-d889-44a2-94c4-f20d52ec3b1f-data\") pod \"postgres-69fb4594cf-pd89t\" (UID: \"759acf50-d889-44a2-94c4-f20d52ec3b1f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"
Jul 26 15:46:14.042794 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:14.042641    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt6fk\" (UniqueName: \"kubernetes.io/projected/759acf50-d889-44a2-94c4-f20d52ec3b1f-kube-api-access-kt6fk\") pod \"postgres-69fb4594cf-pd89t\" (UID: \"759acf50-d889-44a2-94c4-f20d52ec3b1f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"
Jul 26 15:46:14.043018 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:14.042996    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/759acf50-d889-44a2-94c4-f20d52ec3b1f-data\") pod \"postgres-69fb4594cf-pd89t\" (UID: \"759acf50-d889-44a2-94c4-f20d52ec3b1f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"
Jul 26 15:46:14.049998 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:14.049968    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt6fk\" (UniqueName: \"kubernetes.io/projected/759acf50-d889-44a2-94c4-f20d52ec3b1f-kube-api-access-kt6fk\") pod \"postgres-69fb4594cf-pd89t\" (UID: \"759acf50-d889-44a2-94c4-f20d52ec3b1f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"
Jul 26 15:46:14.185480 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:14.185454    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"
Jul 26 15:46:14.310680 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:14.310543    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"]
Jul 26 15:46:14.313411 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:46:14.313378    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759acf50_d889_44a2_94c4_f20d52ec3b1f.slice/crio-bcdf6bdbf659685cb60b0ae1f4f5b8b54d294eb30b20bdd02f826c561c7b0183 WatchSource:0}: Error finding container bcdf6bdbf659685cb60b0ae1f4f5b8b54d294eb30b20bdd02f826c561c7b0183: Status 404 returned error can't find the container with id bcdf6bdbf659685cb60b0ae1f4f5b8b54d294eb30b20bdd02f826c561c7b0183
Jul 26 15:46:14.992425 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:14.992391    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t" event={"ID":"759acf50-d889-44a2-94c4-f20d52ec3b1f","Type":"ContainerStarted","Data":"bcdf6bdbf659685cb60b0ae1f4f5b8b54d294eb30b20bdd02f826c561c7b0183"}
Jul 26 15:46:20.017176 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:20.017147    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t" event={"ID":"759acf50-d889-44a2-94c4-f20d52ec3b1f","Type":"ContainerStarted","Data":"2b7157e267d35c1cfdc3c6fd0ccbf6814d9d22b13c176ae5536a0ef440203e3d"}
Jul 26 15:46:20.017487 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:20.017221    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"
Jul 26 15:46:20.032171 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:20.032110    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t" podStartSLOduration=1.3929835609999999 podStartE2EDuration="7.032093375s" totalImagesPullingTime="5.639109814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:46:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:46:20.030567455 +0000 UTC m=+495.529957293" watchObservedRunningTime="2026-07-26 15:46:20.032093375 +0000 UTC m=+495.531483214"
Jul 26 15:46:26.050521 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:26.050489    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pd89t"
Jul 26 15:46:33.098153 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.098117    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69569df7cf-7vqbj"]
Jul 26 15:46:33.115110 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.115043    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69569df7cf-7vqbj"]
Jul 26 15:46:33.115301 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.115168    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:33.117730 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.117661    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 26 15:46:33.117926 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.117790    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 26 15:46:33.117926 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.117910    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z67mw\""
Jul 26 15:46:33.192745 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.192709    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77894e90-7047-42e0-a0c8-eb61c50bf968-webhook-cert\") pod \"maas-controller-69569df7cf-7vqbj\" (UID: \"77894e90-7047-42e0-a0c8-eb61c50bf968\") " pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:33.192966 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.192797    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtwnh\" (UniqueName: \"kubernetes.io/projected/77894e90-7047-42e0-a0c8-eb61c50bf968-kube-api-access-jtwnh\") pod \"maas-controller-69569df7cf-7vqbj\" (UID: \"77894e90-7047-42e0-a0c8-eb61c50bf968\") " pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:33.259056 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.259016    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86d86cd99-jdjdt"]
Jul 26 15:46:33.261489 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.261465    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:33.273822 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.273792    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d86cd99-jdjdt"]
Jul 26 15:46:33.293598 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.293556    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77894e90-7047-42e0-a0c8-eb61c50bf968-webhook-cert\") pod \"maas-controller-69569df7cf-7vqbj\" (UID: \"77894e90-7047-42e0-a0c8-eb61c50bf968\") " pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:33.293777 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.293634    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtwnh\" (UniqueName: \"kubernetes.io/projected/77894e90-7047-42e0-a0c8-eb61c50bf968-kube-api-access-jtwnh\") pod \"maas-controller-69569df7cf-7vqbj\" (UID: \"77894e90-7047-42e0-a0c8-eb61c50bf968\") " pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:33.293777 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:46:33.293705    2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 26 15:46:33.293859 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:46:33.293778    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77894e90-7047-42e0-a0c8-eb61c50bf968-webhook-cert podName:77894e90-7047-42e0-a0c8-eb61c50bf968 nodeName:}" failed. No retries permitted until 2026-07-26 15:46:33.793761029 +0000 UTC m=+509.293150845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/77894e90-7047-42e0-a0c8-eb61c50bf968-webhook-cert") pod "maas-controller-69569df7cf-7vqbj" (UID: "77894e90-7047-42e0-a0c8-eb61c50bf968") : secret "maas-controller-webhook-cert" not found
Jul 26 15:46:33.301795 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.301770    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtwnh\" (UniqueName: \"kubernetes.io/projected/77894e90-7047-42e0-a0c8-eb61c50bf968-kube-api-access-jtwnh\") pod \"maas-controller-69569df7cf-7vqbj\" (UID: \"77894e90-7047-42e0-a0c8-eb61c50bf968\") " pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:33.395014 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.394963    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4tnk\" (UniqueName: \"kubernetes.io/projected/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-kube-api-access-x4tnk\") pod \"maas-controller-86d86cd99-jdjdt\" (UID: \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\") " pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:33.395185 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.395024    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-webhook-cert\") pod \"maas-controller-86d86cd99-jdjdt\" (UID: \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\") " pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:33.496392 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.496355    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4tnk\" (UniqueName: \"kubernetes.io/projected/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-kube-api-access-x4tnk\") pod \"maas-controller-86d86cd99-jdjdt\" (UID: \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\") " pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:33.496562 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.496401    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-webhook-cert\") pod \"maas-controller-86d86cd99-jdjdt\" (UID: \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\") " pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:33.496562 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:46:33.496524    2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 26 15:46:33.496635 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:46:33.496583    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-webhook-cert podName:d47d2bb4-a8fe-496c-9dca-62ed8fa35950 nodeName:}" failed. No retries permitted until 2026-07-26 15:46:33.996564882 +0000 UTC m=+509.495954698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-webhook-cert") pod "maas-controller-86d86cd99-jdjdt" (UID: "d47d2bb4-a8fe-496c-9dca-62ed8fa35950") : secret "maas-controller-webhook-cert" not found
Jul 26 15:46:33.512543 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.512508    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4tnk\" (UniqueName: \"kubernetes.io/projected/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-kube-api-access-x4tnk\") pod \"maas-controller-86d86cd99-jdjdt\" (UID: \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\") " pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:33.798428 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.798339    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77894e90-7047-42e0-a0c8-eb61c50bf968-webhook-cert\") pod \"maas-controller-69569df7cf-7vqbj\" (UID: \"77894e90-7047-42e0-a0c8-eb61c50bf968\") " pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:33.800708 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:33.800679    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77894e90-7047-42e0-a0c8-eb61c50bf968-webhook-cert\") pod \"maas-controller-69569df7cf-7vqbj\" (UID: \"77894e90-7047-42e0-a0c8-eb61c50bf968\") " pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:34.000069 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:34.000027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-webhook-cert\") pod \"maas-controller-86d86cd99-jdjdt\" (UID: \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\") " pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:34.002399 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:34.002370    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-webhook-cert\") pod \"maas-controller-86d86cd99-jdjdt\" (UID: \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\") " pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:34.027046 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:34.027016    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:34.150373 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:34.150342    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69569df7cf-7vqbj"]
Jul 26 15:46:34.152754 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:46:34.152726    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77894e90_7047_42e0_a0c8_eb61c50bf968.slice/crio-b8457fb161cebe9c5d67bc1d2c93fb4e40f9c72aef6f046d6a33153a0b85c2ab WatchSource:0}: Error finding container b8457fb161cebe9c5d67bc1d2c93fb4e40f9c72aef6f046d6a33153a0b85c2ab: Status 404 returned error can't find the container with id b8457fb161cebe9c5d67bc1d2c93fb4e40f9c72aef6f046d6a33153a0b85c2ab
Jul 26 15:46:34.171095 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:34.171066    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:34.295619 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:34.295590    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d86cd99-jdjdt"]
Jul 26 15:46:34.298960 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:46:34.298931    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47d2bb4_a8fe_496c_9dca_62ed8fa35950.slice/crio-5dd64f16f350cdd6df40de851ff9aaa3574c488c0c1d9fbdf74f79eed50c5d4b WatchSource:0}: Error finding container 5dd64f16f350cdd6df40de851ff9aaa3574c488c0c1d9fbdf74f79eed50c5d4b: Status 404 returned error can't find the container with id 5dd64f16f350cdd6df40de851ff9aaa3574c488c0c1d9fbdf74f79eed50c5d4b
Jul 26 15:46:35.087905 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:35.087849    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69569df7cf-7vqbj" event={"ID":"77894e90-7047-42e0-a0c8-eb61c50bf968","Type":"ContainerStarted","Data":"b8457fb161cebe9c5d67bc1d2c93fb4e40f9c72aef6f046d6a33153a0b85c2ab"}
Jul 26 15:46:35.089557 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:35.089526    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d86cd99-jdjdt" event={"ID":"d47d2bb4-a8fe-496c-9dca-62ed8fa35950","Type":"ContainerStarted","Data":"5dd64f16f350cdd6df40de851ff9aaa3574c488c0c1d9fbdf74f79eed50c5d4b"}
Jul 26 15:46:37.099242 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:37.099141    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d86cd99-jdjdt" event={"ID":"d47d2bb4-a8fe-496c-9dca-62ed8fa35950","Type":"ContainerStarted","Data":"9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679"}
Jul 26 15:46:37.099724 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:37.099246    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:37.100702 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:37.100674    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69569df7cf-7vqbj" event={"ID":"77894e90-7047-42e0-a0c8-eb61c50bf968","Type":"ContainerStarted","Data":"c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7"}
Jul 26 15:46:37.100836 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:37.100819    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:37.115881 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:37.115829    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86d86cd99-jdjdt" podStartSLOduration=1.679831058 podStartE2EDuration="4.115808858s" totalImagesPullingTime="2.4359778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:46:37.114097031 +0000 UTC m=+512.613486872" watchObservedRunningTime="2026-07-26 15:46:37.115808858 +0000 UTC m=+512.615198697"
Jul 26 15:46:37.132261 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:37.132191    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69569df7cf-7vqbj" podStartSLOduration=1.5593262490000002 podStartE2EDuration="4.132141294s" totalImagesPullingTime="2.572815045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:46:37.131784926 +0000 UTC m=+512.631174765" watchObservedRunningTime="2026-07-26 15:46:37.132141294 +0000 UTC m=+512.631531131"
Jul 26 15:46:48.106811 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.106778    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:46:48.107260 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.106848    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:48.158633 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.158605    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69569df7cf-7vqbj"]
Jul 26 15:46:48.158794 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.158770    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-69569df7cf-7vqbj" podUID="77894e90-7047-42e0-a0c8-eb61c50bf968" containerName="manager" containerID="cri-o://c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7" gracePeriod=30
Jul 26 15:46:48.417647 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.417618    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:48.513341 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.513309    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77894e90-7047-42e0-a0c8-eb61c50bf968-webhook-cert\") pod \"77894e90-7047-42e0-a0c8-eb61c50bf968\" (UID: \"77894e90-7047-42e0-a0c8-eb61c50bf968\") "
Jul 26 15:46:48.513469 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.513358    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtwnh\" (UniqueName: \"kubernetes.io/projected/77894e90-7047-42e0-a0c8-eb61c50bf968-kube-api-access-jtwnh\") pod \"77894e90-7047-42e0-a0c8-eb61c50bf968\" (UID: \"77894e90-7047-42e0-a0c8-eb61c50bf968\") "
Jul 26 15:46:48.515525 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.515496    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77894e90-7047-42e0-a0c8-eb61c50bf968-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "77894e90-7047-42e0-a0c8-eb61c50bf968" (UID: "77894e90-7047-42e0-a0c8-eb61c50bf968"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:46:48.515525 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.515491    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77894e90-7047-42e0-a0c8-eb61c50bf968-kube-api-access-jtwnh" (OuterVolumeSpecName: "kube-api-access-jtwnh") pod "77894e90-7047-42e0-a0c8-eb61c50bf968" (UID: "77894e90-7047-42e0-a0c8-eb61c50bf968"). InnerVolumeSpecName "kube-api-access-jtwnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:46:48.619738 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.615283    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77894e90-7047-42e0-a0c8-eb61c50bf968-webhook-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:46:48.619738 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:48.615320    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtwnh\" (UniqueName: \"kubernetes.io/projected/77894e90-7047-42e0-a0c8-eb61c50bf968-kube-api-access-jtwnh\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:46:49.147857 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:49.147830    2560 generic.go:356] "Generic (PLEG): container finished" podID="77894e90-7047-42e0-a0c8-eb61c50bf968" containerID="c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7" exitCode=0
Jul 26 15:46:49.148230 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:49.147880    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69569df7cf-7vqbj"
Jul 26 15:46:49.148230 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:49.147910    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69569df7cf-7vqbj" event={"ID":"77894e90-7047-42e0-a0c8-eb61c50bf968","Type":"ContainerDied","Data":"c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7"}
Jul 26 15:46:49.148230 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:49.147947    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69569df7cf-7vqbj" event={"ID":"77894e90-7047-42e0-a0c8-eb61c50bf968","Type":"ContainerDied","Data":"b8457fb161cebe9c5d67bc1d2c93fb4e40f9c72aef6f046d6a33153a0b85c2ab"}
Jul 26 15:46:49.148230 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:49.147972    2560 scope.go:117] "RemoveContainer" containerID="c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7"
Jul 26 15:46:49.157439 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:49.157423    2560 scope.go:117] "RemoveContainer" containerID="c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7"
Jul 26 15:46:49.157682 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:46:49.157663    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7\": container with ID starting with c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7 not found: ID does not exist" containerID="c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7"
Jul 26 15:46:49.157725 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:49.157689    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7"} err="failed to get container status \"c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7\": rpc error: code = NotFound desc = could not find container \"c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7\": container with ID starting with c2adc981a2597d3c7279ed517f79ba91476e4a65c71dfe9aff2f5864dc5ddad7 not found: ID does not exist"
Jul 26 15:46:49.164053 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:49.164017    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69569df7cf-7vqbj"]
Jul 26 15:46:49.165796 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:49.165775    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-69569df7cf-7vqbj"]
Jul 26 15:46:51.137946 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:46:51.137913    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77894e90-7047-42e0-a0c8-eb61c50bf968" path="/var/lib/kubelet/pods/77894e90-7047-42e0-a0c8-eb61c50bf968/volumes"
Jul 26 15:47:10.396540 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.396506    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"]
Jul 26 15:47:10.396918 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.396844    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77894e90-7047-42e0-a0c8-eb61c50bf968" containerName="manager"
Jul 26 15:47:10.396918 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.396861    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="77894e90-7047-42e0-a0c8-eb61c50bf968" containerName="manager"
Jul 26 15:47:10.397014 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.396940    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="77894e90-7047-42e0-a0c8-eb61c50bf968" containerName="manager"
Jul 26 15:47:10.403444 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.403427    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.406210 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.406189    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ps89k\""
Jul 26 15:47:10.406339 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.406193    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 26 15:47:10.409850 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.409750    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"]
Jul 26 15:47:10.579976 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.579952    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhsm6\" (UniqueName: \"kubernetes.io/projected/9c444d25-e535-4332-9d97-14733da7bdcc-kube-api-access-zhsm6\") pod \"maas-api-c56dc8ff8-pz8zn\" (UID: \"9c444d25-e535-4332-9d97-14733da7bdcc\") " pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.579976 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.579984    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c444d25-e535-4332-9d97-14733da7bdcc-openshift-service-ca\") pod \"maas-api-c56dc8ff8-pz8zn\" (UID: \"9c444d25-e535-4332-9d97-14733da7bdcc\") " pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.580148 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.580027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c444d25-e535-4332-9d97-14733da7bdcc-maas-api-tls\") pod \"maas-api-c56dc8ff8-pz8zn\" (UID: \"9c444d25-e535-4332-9d97-14733da7bdcc\") " pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.681334 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.681265    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c444d25-e535-4332-9d97-14733da7bdcc-maas-api-tls\") pod \"maas-api-c56dc8ff8-pz8zn\" (UID: \"9c444d25-e535-4332-9d97-14733da7bdcc\") " pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.681477 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.681341    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhsm6\" (UniqueName: \"kubernetes.io/projected/9c444d25-e535-4332-9d97-14733da7bdcc-kube-api-access-zhsm6\") pod \"maas-api-c56dc8ff8-pz8zn\" (UID: \"9c444d25-e535-4332-9d97-14733da7bdcc\") " pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.681477 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.681372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c444d25-e535-4332-9d97-14733da7bdcc-openshift-service-ca\") pod \"maas-api-c56dc8ff8-pz8zn\" (UID: \"9c444d25-e535-4332-9d97-14733da7bdcc\") " pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.682015 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.681995    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c444d25-e535-4332-9d97-14733da7bdcc-openshift-service-ca\") pod \"maas-api-c56dc8ff8-pz8zn\" (UID: \"9c444d25-e535-4332-9d97-14733da7bdcc\") " pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.683645 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.683622    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c444d25-e535-4332-9d97-14733da7bdcc-maas-api-tls\") pod \"maas-api-c56dc8ff8-pz8zn\" (UID: \"9c444d25-e535-4332-9d97-14733da7bdcc\") " pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.689597 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.689573    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhsm6\" (UniqueName: \"kubernetes.io/projected/9c444d25-e535-4332-9d97-14733da7bdcc-kube-api-access-zhsm6\") pod \"maas-api-c56dc8ff8-pz8zn\" (UID: \"9c444d25-e535-4332-9d97-14733da7bdcc\") " pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.715321 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.715299    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:10.852980 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.852954    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"]
Jul 26 15:47:10.855232 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:47:10.855203    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c444d25_e535_4332_9d97_14733da7bdcc.slice/crio-7d5ba90f66c51b477debce5a614de368262cd17591310ce5b943554faeeb7b34 WatchSource:0}: Error finding container 7d5ba90f66c51b477debce5a614de368262cd17591310ce5b943554faeeb7b34: Status 404 returned error can't find the container with id 7d5ba90f66c51b477debce5a614de368262cd17591310ce5b943554faeeb7b34
Jul 26 15:47:10.980792 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.980717    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"]
Jul 26 15:47:10.986661 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.986637    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"
Jul 26 15:47:10.989078 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.989057    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4kr9l\""
Jul 26 15:47:10.989254 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.989235    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 26 15:47:10.996012 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:10.995993    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"]
Jul 26 15:47:11.000378 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.000359    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-pnh69"]
Jul 26 15:47:11.004385 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.004368    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69"
Jul 26 15:47:11.014614 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.014588    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-pnh69"]
Jul 26 15:47:11.086416 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.086384    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8g7w\" (UniqueName: \"kubernetes.io/projected/fff00ece-1226-40dd-bf8d-71c600041701-kube-api-access-m8g7w\") pod \"payload-pre-processing-687dbb6cf6-64tt4\" (UID: \"fff00ece-1226-40dd-bf8d-71c600041701\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"
Jul 26 15:47:11.086416 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.086424    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff00ece-1226-40dd-bf8d-71c600041701-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-64tt4\" (UID: \"fff00ece-1226-40dd-bf8d-71c600041701\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"
Jul 26 15:47:11.187019 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.186987    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1a3706c-d0ff-4751-8639-a1de8788e821-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-pnh69\" (UID: \"f1a3706c-d0ff-4751-8639-a1de8788e821\") " pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69"
Jul 26 15:47:11.187019 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.187024    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8g7w\" (UniqueName: \"kubernetes.io/projected/fff00ece-1226-40dd-bf8d-71c600041701-kube-api-access-m8g7w\") pod \"payload-pre-processing-687dbb6cf6-64tt4\" (UID: \"fff00ece-1226-40dd-bf8d-71c600041701\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"
Jul 26 15:47:11.187239 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.187046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp7c4\" (UniqueName: \"kubernetes.io/projected/f1a3706c-d0ff-4751-8639-a1de8788e821-kube-api-access-zp7c4\") pod \"payload-processing-5cc6cd444b-pnh69\" (UID: \"f1a3706c-d0ff-4751-8639-a1de8788e821\") " pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69"
Jul 26 15:47:11.187239 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.187100    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff00ece-1226-40dd-bf8d-71c600041701-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-64tt4\" (UID: \"fff00ece-1226-40dd-bf8d-71c600041701\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"
Jul 26 15:47:11.187835 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.187816    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff00ece-1226-40dd-bf8d-71c600041701-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-64tt4\" (UID: \"fff00ece-1226-40dd-bf8d-71c600041701\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"
Jul 26 15:47:11.194273 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.194244    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8g7w\" (UniqueName: \"kubernetes.io/projected/fff00ece-1226-40dd-bf8d-71c600041701-kube-api-access-m8g7w\") pod \"payload-pre-processing-687dbb6cf6-64tt4\" (UID: \"fff00ece-1226-40dd-bf8d-71c600041701\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"
Jul 26 15:47:11.236671 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.236583    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn" event={"ID":"9c444d25-e535-4332-9d97-14733da7bdcc","Type":"ContainerStarted","Data":"7d5ba90f66c51b477debce5a614de368262cd17591310ce5b943554faeeb7b34"}
Jul 26 15:47:11.287538 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.287505    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1a3706c-d0ff-4751-8639-a1de8788e821-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-pnh69\" (UID: \"f1a3706c-d0ff-4751-8639-a1de8788e821\") " pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69"
Jul 26 15:47:11.287538 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.287542    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp7c4\" (UniqueName: \"kubernetes.io/projected/f1a3706c-d0ff-4751-8639-a1de8788e821-kube-api-access-zp7c4\") pod \"payload-processing-5cc6cd444b-pnh69\" (UID: \"f1a3706c-d0ff-4751-8639-a1de8788e821\") " pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69"
Jul 26 15:47:11.288261 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.288239    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1a3706c-d0ff-4751-8639-a1de8788e821-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-pnh69\" (UID: \"f1a3706c-d0ff-4751-8639-a1de8788e821\") " pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69"
Jul 26 15:47:11.297813 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.297787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp7c4\" (UniqueName: \"kubernetes.io/projected/f1a3706c-d0ff-4751-8639-a1de8788e821-kube-api-access-zp7c4\") pod \"payload-processing-5cc6cd444b-pnh69\" (UID: \"f1a3706c-d0ff-4751-8639-a1de8788e821\") " pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69"
Jul 26 15:47:11.297962 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.297914    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"
Jul 26 15:47:11.315330 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.315304    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69"
Jul 26 15:47:11.438065 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.438034    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"]
Jul 26 15:47:11.441128 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:47:11.441084    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff00ece_1226_40dd_bf8d_71c600041701.slice/crio-6ffbaaa90f37da86b909a684bee9b99913b47ae6bfae75c7f65ff3f2fab3900c WatchSource:0}: Error finding container 6ffbaaa90f37da86b909a684bee9b99913b47ae6bfae75c7f65ff3f2fab3900c: Status 404 returned error can't find the container with id 6ffbaaa90f37da86b909a684bee9b99913b47ae6bfae75c7f65ff3f2fab3900c
Jul 26 15:47:11.463365 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:11.463331    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-pnh69"]
Jul 26 15:47:11.466685 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:47:11.466661    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a3706c_d0ff_4751_8639_a1de8788e821.slice/crio-d2d4a515ecf81a06bb93590d09dd7f2cae9f817674991a8475784c1689d0c7ad WatchSource:0}: Error finding container d2d4a515ecf81a06bb93590d09dd7f2cae9f817674991a8475784c1689d0c7ad: Status 404 returned error can't find the container with id d2d4a515ecf81a06bb93590d09dd7f2cae9f817674991a8475784c1689d0c7ad
Jul 26 15:47:12.242559 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:12.242523    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69" event={"ID":"f1a3706c-d0ff-4751-8639-a1de8788e821","Type":"ContainerStarted","Data":"d2d4a515ecf81a06bb93590d09dd7f2cae9f817674991a8475784c1689d0c7ad"}
Jul 26 15:47:12.243590 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:12.243564    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4" event={"ID":"fff00ece-1226-40dd-bf8d-71c600041701","Type":"ContainerStarted","Data":"6ffbaaa90f37da86b909a684bee9b99913b47ae6bfae75c7f65ff3f2fab3900c"}
Jul 26 15:47:13.249975 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:13.249942    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn" event={"ID":"9c444d25-e535-4332-9d97-14733da7bdcc","Type":"ContainerStarted","Data":"6b66b4aff1b4ba3f85c1a29ee7705e124303c886a77f099d7626169aeb5eb7d9"}
Jul 26 15:47:13.250358 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:13.250086    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:13.269300 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:13.269242    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn" podStartSLOduration=1.876909789 podStartE2EDuration="3.269226555s" totalImagesPullingTime="1.392316766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:47:13.266032972 +0000 UTC m=+548.765422812" watchObservedRunningTime="2026-07-26 15:47:13.269226555 +0000 UTC m=+548.768616375"
Jul 26 15:47:15.261035 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:15.260996    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69" event={"ID":"f1a3706c-d0ff-4751-8639-a1de8788e821","Type":"ContainerStarted","Data":"45076cf02d0df7af4c21c5bdb9c1db5d7c546ba75b2dbf56ab940ad6111a8b71"}
Jul 26 15:47:15.261473 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:15.261134    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69"
Jul 26 15:47:15.262436 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:15.262415    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4" event={"ID":"fff00ece-1226-40dd-bf8d-71c600041701","Type":"ContainerStarted","Data":"d2a8d0ac1e1638036383dcbf81983bb591b19824dd0adc047dfc932c275e1812"}
Jul 26 15:47:15.262560 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:15.262540    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"
Jul 26 15:47:15.276369 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:15.276330    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69" podStartSLOduration=2.497077525 podStartE2EDuration="5.276316276s" totalImagesPullingTime="2.779238751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:47:15.274764191 +0000 UTC m=+550.774154029" watchObservedRunningTime="2026-07-26 15:47:15.276316276 +0000 UTC m=+550.775706091"
Jul 26 15:47:15.290244 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:15.290207    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4" podStartSLOduration=2.4737953 podStartE2EDuration="5.290194181s" totalImagesPullingTime="2.816398881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:47:15.288957284 +0000 UTC m=+550.788347123" watchObservedRunningTime="2026-07-26 15:47:15.290194181 +0000 UTC m=+550.789584019"
Jul 26 15:47:19.263983 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:19.263952    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c56dc8ff8-pz8zn"
Jul 26 15:47:26.268420 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:26.268392    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-64tt4"
Jul 26 15:47:26.268780 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:26.268460    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-pnh69"
Jul 26 15:47:58.564142 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.564102    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"]
Jul 26 15:47:58.568952 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.568932    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.572049 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.572028    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 26 15:47:58.572149 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.572030    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6w9zr\""
Jul 26 15:47:58.572313 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.572296    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 26 15:47:58.572386 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.572316    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 26 15:47:58.577485 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.577462    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"]
Jul 26 15:47:58.675081 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.675050    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.675219 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.675094    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c907832-a817-43a9-8284-f91e72c6da3e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.675219 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.675122    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.675219 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.675137    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72j55\" (UniqueName: \"kubernetes.io/projected/5c907832-a817-43a9-8284-f91e72c6da3e-kube-api-access-72j55\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.675219 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.675156    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.675219 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.675173    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.776021 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.775988    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.776211 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.776047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c907832-a817-43a9-8284-f91e72c6da3e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.776211 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.776081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.776211 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.776105    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72j55\" (UniqueName: \"kubernetes.io/projected/5c907832-a817-43a9-8284-f91e72c6da3e-kube-api-access-72j55\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.776211 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.776127    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.776211 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.776152    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.776553 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.776519    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.776669 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.776582    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.776669 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.776590    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.778159 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.778144    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c907832-a817-43a9-8284-f91e72c6da3e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.778568 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.778547    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c907832-a817-43a9-8284-f91e72c6da3e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.784426 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.784404    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72j55\" (UniqueName: \"kubernetes.io/projected/5c907832-a817-43a9-8284-f91e72c6da3e-kube-api-access-72j55\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn\" (UID: \"5c907832-a817-43a9-8284-f91e72c6da3e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:58.879399 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:58.879325    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:47:59.008649 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:59.008622    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"]
Jul 26 15:47:59.011174 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:47:59.011141    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c907832_a817_43a9_8284_f91e72c6da3e.slice/crio-57b8b88a96328c6fd26140ce8c0134e328a9853a2ff2cad0df4caa0f7a9185d1 WatchSource:0}: Error finding container 57b8b88a96328c6fd26140ce8c0134e328a9853a2ff2cad0df4caa0f7a9185d1: Status 404 returned error can't find the container with id 57b8b88a96328c6fd26140ce8c0134e328a9853a2ff2cad0df4caa0f7a9185d1
Jul 26 15:47:59.446353 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:47:59.446315    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn" event={"ID":"5c907832-a817-43a9-8284-f91e72c6da3e","Type":"ContainerStarted","Data":"57b8b88a96328c6fd26140ce8c0134e328a9853a2ff2cad0df4caa0f7a9185d1"}
Jul 26 15:48:00.451577 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:48:00.451539    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn" event={"ID":"5c907832-a817-43a9-8284-f91e72c6da3e","Type":"ContainerStarted","Data":"9ec41052fdcbba8245f28dd9a24cc72e13749d652e13ba7b56fe2ca0dd6c5921"}
Jul 26 15:48:00.468266 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:48:00.468215    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn" podStartSLOduration=1.53102341 podStartE2EDuration="2.468198145s" totalImagesPullingTime="937.174735ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:48:00.466343658 +0000 UTC m=+595.965733501" watchObservedRunningTime="2026-07-26 15:48:00.468198145 +0000 UTC m=+595.967588032"
Jul 26 15:48:08.879838 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:48:08.879795    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:48:08.880335 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:48:08.879863    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:48:08.892593 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:48:08.892560    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:48:09.503211 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:48:09.503184    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn"
Jul 26 15:49:47.828829 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:47.828789    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86d86cd99-jdjdt"]
Jul 26 15:49:47.829355 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:47.829127    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-86d86cd99-jdjdt" podUID="d47d2bb4-a8fe-496c-9dca-62ed8fa35950" containerName="manager" containerID="cri-o://9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679" gracePeriod=30
Jul 26 15:49:48.065719 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.065695    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:49:48.202518 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.202492    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-webhook-cert\") pod \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\" (UID: \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\") "
Jul 26 15:49:48.202652 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.202540    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4tnk\" (UniqueName: \"kubernetes.io/projected/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-kube-api-access-x4tnk\") pod \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\" (UID: \"d47d2bb4-a8fe-496c-9dca-62ed8fa35950\") "
Jul 26 15:49:48.204353 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.204322    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d47d2bb4-a8fe-496c-9dca-62ed8fa35950" (UID: "d47d2bb4-a8fe-496c-9dca-62ed8fa35950"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:49:48.204449 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.204380    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-kube-api-access-x4tnk" (OuterVolumeSpecName: "kube-api-access-x4tnk") pod "d47d2bb4-a8fe-496c-9dca-62ed8fa35950" (UID: "d47d2bb4-a8fe-496c-9dca-62ed8fa35950"). InnerVolumeSpecName "kube-api-access-x4tnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:49:48.302977 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.302946    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-webhook-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:49:48.302977 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.302975    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4tnk\" (UniqueName: \"kubernetes.io/projected/d47d2bb4-a8fe-496c-9dca-62ed8fa35950-kube-api-access-x4tnk\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:49:48.905533 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.905501    2560 generic.go:356] "Generic (PLEG): container finished" podID="d47d2bb4-a8fe-496c-9dca-62ed8fa35950" containerID="9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679" exitCode=0
Jul 26 15:49:48.905880 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.905540    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d86cd99-jdjdt"
Jul 26 15:49:48.905880 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.905588    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d86cd99-jdjdt" event={"ID":"d47d2bb4-a8fe-496c-9dca-62ed8fa35950","Type":"ContainerDied","Data":"9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679"}
Jul 26 15:49:48.905880 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.905638    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d86cd99-jdjdt" event={"ID":"d47d2bb4-a8fe-496c-9dca-62ed8fa35950","Type":"ContainerDied","Data":"5dd64f16f350cdd6df40de851ff9aaa3574c488c0c1d9fbdf74f79eed50c5d4b"}
Jul 26 15:49:48.905880 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.905666    2560 scope.go:117] "RemoveContainer" containerID="9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679"
Jul 26 15:49:48.914626 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.914609    2560 scope.go:117] "RemoveContainer" containerID="9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679"
Jul 26 15:49:48.914876 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:49:48.914857    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679\": container with ID starting with 9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679 not found: ID does not exist" containerID="9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679"
Jul 26 15:49:48.914940 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.914885    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679"} err="failed to get container status \"9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679\": rpc error: code = NotFound desc = could not find container \"9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679\": container with ID starting with 9a6b77d4d038f5e4fec7245bd9d9f96e10b08645c2fc4249048c224eb0f8c679 not found: ID does not exist"
Jul 26 15:49:48.927952 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.927932    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86d86cd99-jdjdt"]
Jul 26 15:49:48.931625 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:48.931604    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86d86cd99-jdjdt"]
Jul 26 15:49:49.137675 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:49:49.137648    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d47d2bb4-a8fe-496c-9dca-62ed8fa35950" path="/var/lib/kubelet/pods/d47d2bb4-a8fe-496c-9dca-62ed8fa35950/volumes"
Jul 26 15:50:49.710494 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.710458    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86d86cd99-fdmrd"]
Jul 26 15:50:49.711041 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.710907    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47d2bb4-a8fe-496c-9dca-62ed8fa35950" containerName="manager"
Jul 26 15:50:49.711041 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.710927    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47d2bb4-a8fe-496c-9dca-62ed8fa35950" containerName="manager"
Jul 26 15:50:49.711041 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.711031    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d47d2bb4-a8fe-496c-9dca-62ed8fa35950" containerName="manager"
Jul 26 15:50:49.714100 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.714077    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:50:49.716227 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.716205    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 26 15:50:49.716755 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.716736    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 26 15:50:49.716842 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.716759    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z67mw\""
Jul 26 15:50:49.722286 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.722264    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d86cd99-fdmrd"]
Jul 26 15:50:49.865078 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.865053    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-webhook-cert\") pod \"maas-controller-86d86cd99-fdmrd\" (UID: \"3a14b1ea-4f6e-4147-b6f5-da7af19201a5\") " pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:50:49.865193 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.865091    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsngw\" (UniqueName: \"kubernetes.io/projected/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-kube-api-access-jsngw\") pod \"maas-controller-86d86cd99-fdmrd\" (UID: \"3a14b1ea-4f6e-4147-b6f5-da7af19201a5\") " pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:50:49.966034 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.965980    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-webhook-cert\") pod \"maas-controller-86d86cd99-fdmrd\" (UID: \"3a14b1ea-4f6e-4147-b6f5-da7af19201a5\") " pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:50:49.966034 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.966019    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsngw\" (UniqueName: \"kubernetes.io/projected/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-kube-api-access-jsngw\") pod \"maas-controller-86d86cd99-fdmrd\" (UID: \"3a14b1ea-4f6e-4147-b6f5-da7af19201a5\") " pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:50:49.968305 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.968279    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-webhook-cert\") pod \"maas-controller-86d86cd99-fdmrd\" (UID: \"3a14b1ea-4f6e-4147-b6f5-da7af19201a5\") " pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:50:49.973696 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:49.973676    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsngw\" (UniqueName: \"kubernetes.io/projected/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-kube-api-access-jsngw\") pod \"maas-controller-86d86cd99-fdmrd\" (UID: \"3a14b1ea-4f6e-4147-b6f5-da7af19201a5\") " pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:50:50.024512 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:50.024489    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:50:50.153151 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:50.153127    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d86cd99-fdmrd"]
Jul 26 15:50:50.155860 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:50:50.155827    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a14b1ea_4f6e_4147_b6f5_da7af19201a5.slice/crio-665ce45f7e5fe4463f9156366690e21237cf3f3548033c05c6c81c8bad799bd2 WatchSource:0}: Error finding container 665ce45f7e5fe4463f9156366690e21237cf3f3548033c05c6c81c8bad799bd2: Status 404 returned error can't find the container with id 665ce45f7e5fe4463f9156366690e21237cf3f3548033c05c6c81c8bad799bd2
Jul 26 15:50:50.157164 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:50.157146    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:50:51.164478 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:51.164434    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d86cd99-fdmrd" event={"ID":"3a14b1ea-4f6e-4147-b6f5-da7af19201a5","Type":"ContainerStarted","Data":"aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73"}
Jul 26 15:50:51.164478 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:51.164480    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d86cd99-fdmrd" event={"ID":"3a14b1ea-4f6e-4147-b6f5-da7af19201a5","Type":"ContainerStarted","Data":"665ce45f7e5fe4463f9156366690e21237cf3f3548033c05c6c81c8bad799bd2"}
Jul 26 15:50:51.164872 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:51.164578    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:50:51.182112 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:50:51.182073    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86d86cd99-fdmrd" podStartSLOduration=1.765450665 podStartE2EDuration="2.182060268s" totalImagesPullingTime="416.609603ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:50:51.18012282 +0000 UTC m=+766.679512658" watchObservedRunningTime="2026-07-26 15:50:51.182060268 +0000 UTC m=+766.681450106"
Jul 26 15:51:02.171298 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:02.171270    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:51:06.802450 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:06.802412    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86d86cd99-fdmrd"]
Jul 26 15:51:06.802870 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:06.802601    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-86d86cd99-fdmrd" podUID="3a14b1ea-4f6e-4147-b6f5-da7af19201a5" containerName="manager" containerID="cri-o://aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73" gracePeriod=30
Jul 26 15:51:07.046832 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.046808    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:51:07.181469 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.181446    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-webhook-cert\") pod \"3a14b1ea-4f6e-4147-b6f5-da7af19201a5\" (UID: \"3a14b1ea-4f6e-4147-b6f5-da7af19201a5\") "
Jul 26 15:51:07.181621 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.181495    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsngw\" (UniqueName: \"kubernetes.io/projected/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-kube-api-access-jsngw\") pod \"3a14b1ea-4f6e-4147-b6f5-da7af19201a5\" (UID: \"3a14b1ea-4f6e-4147-b6f5-da7af19201a5\") "
Jul 26 15:51:07.183444 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.183416    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3a14b1ea-4f6e-4147-b6f5-da7af19201a5" (UID: "3a14b1ea-4f6e-4147-b6f5-da7af19201a5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:51:07.183536 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.183420    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-kube-api-access-jsngw" (OuterVolumeSpecName: "kube-api-access-jsngw") pod "3a14b1ea-4f6e-4147-b6f5-da7af19201a5" (UID: "3a14b1ea-4f6e-4147-b6f5-da7af19201a5"). InnerVolumeSpecName "kube-api-access-jsngw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:51:07.228307 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.228280    2560 generic.go:356] "Generic (PLEG): container finished" podID="3a14b1ea-4f6e-4147-b6f5-da7af19201a5" containerID="aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73" exitCode=0
Jul 26 15:51:07.228391 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.228322    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d86cd99-fdmrd"
Jul 26 15:51:07.228391 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.228333    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d86cd99-fdmrd" event={"ID":"3a14b1ea-4f6e-4147-b6f5-da7af19201a5","Type":"ContainerDied","Data":"aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73"}
Jul 26 15:51:07.228391 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.228379    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d86cd99-fdmrd" event={"ID":"3a14b1ea-4f6e-4147-b6f5-da7af19201a5","Type":"ContainerDied","Data":"665ce45f7e5fe4463f9156366690e21237cf3f3548033c05c6c81c8bad799bd2"}
Jul 26 15:51:07.228492 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.228402    2560 scope.go:117] "RemoveContainer" containerID="aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73"
Jul 26 15:51:07.235911 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.235869    2560 scope.go:117] "RemoveContainer" containerID="aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73"
Jul 26 15:51:07.236235 ip-10-0-132-10 kubenswrapper[2560]: E0726 15:51:07.236211    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73\": container with ID starting with aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73 not found: ID does not exist" containerID="aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73"
Jul 26 15:51:07.236304 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.236239    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73"} err="failed to get container status \"aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73\": rpc error: code = NotFound desc = could not find container \"aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73\": container with ID starting with aba78d0a73a34f2be7fdaf9556c5c9fe494511ad581fe44b6065023e75438e73 not found: ID does not exist"
Jul 26 15:51:07.248095 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.248077    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-86d86cd99-fdmrd"]
Jul 26 15:51:07.252504 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.252485    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-86d86cd99-fdmrd"]
Jul 26 15:51:07.282267 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.282249    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-webhook-cert\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:51:07.282267 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:07.282266    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsngw\" (UniqueName: \"kubernetes.io/projected/3a14b1ea-4f6e-4147-b6f5-da7af19201a5-kube-api-access-jsngw\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 15:51:08.010458 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.010431    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86d86cd99-sm5c5"]
Jul 26 15:51:08.010819 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.010748    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a14b1ea-4f6e-4147-b6f5-da7af19201a5" containerName="manager"
Jul 26 15:51:08.010819 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.010759    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a14b1ea-4f6e-4147-b6f5-da7af19201a5" containerName="manager"
Jul 26 15:51:08.010819 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.010818    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a14b1ea-4f6e-4147-b6f5-da7af19201a5" containerName="manager"
Jul 26 15:51:08.014900 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.014870    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d86cd99-sm5c5"
Jul 26 15:51:08.017657 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.017634    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z67mw\""
Jul 26 15:51:08.017772 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.017675    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 26 15:51:08.017772 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.017720    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 26 15:51:08.023446 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.023423    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d86cd99-sm5c5"]
Jul 26 15:51:08.086993 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.086971    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bp2d\" (UniqueName: \"kubernetes.io/projected/9e01530d-9d12-4e71-b9d1-a311b268f572-kube-api-access-4bp2d\") pod \"maas-controller-86d86cd99-sm5c5\" (UID: \"9e01530d-9d12-4e71-b9d1-a311b268f572\") " pod="opendatahub/maas-controller-86d86cd99-sm5c5"
Jul 26 15:51:08.087105 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.087027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e01530d-9d12-4e71-b9d1-a311b268f572-webhook-cert\") pod \"maas-controller-86d86cd99-sm5c5\" (UID: \"9e01530d-9d12-4e71-b9d1-a311b268f572\") " pod="opendatahub/maas-controller-86d86cd99-sm5c5"
Jul 26 15:51:08.188346 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.188327    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e01530d-9d12-4e71-b9d1-a311b268f572-webhook-cert\") pod \"maas-controller-86d86cd99-sm5c5\" (UID: \"9e01530d-9d12-4e71-b9d1-a311b268f572\") " pod="opendatahub/maas-controller-86d86cd99-sm5c5"
Jul 26 15:51:08.188459 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.188386    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bp2d\" (UniqueName: \"kubernetes.io/projected/9e01530d-9d12-4e71-b9d1-a311b268f572-kube-api-access-4bp2d\") pod \"maas-controller-86d86cd99-sm5c5\" (UID: \"9e01530d-9d12-4e71-b9d1-a311b268f572\") " pod="opendatahub/maas-controller-86d86cd99-sm5c5"
Jul 26 15:51:08.190604 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.190580    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e01530d-9d12-4e71-b9d1-a311b268f572-webhook-cert\") pod \"maas-controller-86d86cd99-sm5c5\" (UID: \"9e01530d-9d12-4e71-b9d1-a311b268f572\") " pod="opendatahub/maas-controller-86d86cd99-sm5c5"
Jul 26 15:51:08.197041 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.197017    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bp2d\" (UniqueName: \"kubernetes.io/projected/9e01530d-9d12-4e71-b9d1-a311b268f572-kube-api-access-4bp2d\") pod \"maas-controller-86d86cd99-sm5c5\" (UID: \"9e01530d-9d12-4e71-b9d1-a311b268f572\") " pod="opendatahub/maas-controller-86d86cd99-sm5c5"
Jul 26 15:51:08.325183 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.325128    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86d86cd99-sm5c5"
Jul 26 15:51:08.447569 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:08.447539    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86d86cd99-sm5c5"]
Jul 26 15:51:08.449385 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:51:08.449352    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e01530d_9d12_4e71_b9d1_a311b268f572.slice/crio-9e2f5e232890a7e691ebaceae07408922c853f85c55abbd627a5bf7170430db0 WatchSource:0}: Error finding container 9e2f5e232890a7e691ebaceae07408922c853f85c55abbd627a5bf7170430db0: Status 404 returned error can't find the container with id 9e2f5e232890a7e691ebaceae07408922c853f85c55abbd627a5bf7170430db0
Jul 26 15:51:09.139623 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:09.139586    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a14b1ea-4f6e-4147-b6f5-da7af19201a5" path="/var/lib/kubelet/pods/3a14b1ea-4f6e-4147-b6f5-da7af19201a5/volumes"
Jul 26 15:51:09.243325 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:09.243291    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d86cd99-sm5c5" event={"ID":"9e01530d-9d12-4e71-b9d1-a311b268f572","Type":"ContainerStarted","Data":"5a1b96e1a4fc986208e4f0e4b56f976275ed942b55dd3c7374a4807709f825cf"}
Jul 26 15:51:09.243457 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:09.243343    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86d86cd99-sm5c5" event={"ID":"9e01530d-9d12-4e71-b9d1-a311b268f572","Type":"ContainerStarted","Data":"9e2f5e232890a7e691ebaceae07408922c853f85c55abbd627a5bf7170430db0"}
Jul 26 15:51:09.243457 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:09.243419    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86d86cd99-sm5c5"
Jul 26 15:51:09.262022 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:09.261972    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-86d86cd99-sm5c5" podStartSLOduration=1.96143621 podStartE2EDuration="2.261956032s" totalImagesPullingTime="300.519822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:51:09.260469431 +0000 UTC m=+784.759859271" watchObservedRunningTime="2026-07-26 15:51:09.261956032 +0000 UTC m=+784.761345870"
Jul 26 15:51:20.249506 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:51:20.249479    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86d86cd99-sm5c5"
Jul 26 15:59:43.162174 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.162142    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"]
Jul 26 15:59:43.165660 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.165638    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.168042 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.168021    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-25ae8ef7-kserve-self-signed-certs\""
Jul 26 15:59:43.176366 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.176342    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"]
Jul 26 15:59:43.240467 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.240441    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-tmp-dir\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.240591 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.240470    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-home\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.240591 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.240494    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pq5m\" (UniqueName: \"kubernetes.io/projected/54b34042-382b-47e0-a65b-4fffd41d0a43-kube-api-access-5pq5m\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.240700 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.240604    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-dshm\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.240700 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.240672    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54b34042-382b-47e0-a65b-4fffd41d0a43-tls-certs\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.240774 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.240716    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-model-cache\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.341167 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.341138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-model-cache\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.341303 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.341183    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-tmp-dir\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.341303 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.341203    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-home\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.341303 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.341218    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pq5m\" (UniqueName: \"kubernetes.io/projected/54b34042-382b-47e0-a65b-4fffd41d0a43-kube-api-access-5pq5m\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.341303 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.341243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-dshm\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.341303 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.341276    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54b34042-382b-47e0-a65b-4fffd41d0a43-tls-certs\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.341549 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.341486    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-model-cache\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.341605 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.341546    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-home\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.341605 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.341565    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-tmp-dir\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.343577 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.343545    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-dshm\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.343738 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.343714    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54b34042-382b-47e0-a65b-4fffd41d0a43-tls-certs\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.350288 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.350268    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pq5m\" (UniqueName: \"kubernetes.io/projected/54b34042-382b-47e0-a65b-4fffd41d0a43-kube-api-access-5pq5m\") pod \"e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") " pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.477594 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.477522    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:43.603495 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:43.603472    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"]
Jul 26 15:59:43.605771 ip-10-0-132-10 kubenswrapper[2560]: W0726 15:59:43.605743    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54b34042_382b_47e0_a65b_4fffd41d0a43.slice/crio-fb7420786b70b61f721d59f0c187801633679ae96a4809eaa2bc4fa69704c3d2 WatchSource:0}: Error finding container fb7420786b70b61f721d59f0c187801633679ae96a4809eaa2bc4fa69704c3d2: Status 404 returned error can't find the container with id fb7420786b70b61f721d59f0c187801633679ae96a4809eaa2bc4fa69704c3d2
Jul 26 15:59:44.341570 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:44.341531    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk" event={"ID":"54b34042-382b-47e0-a65b-4fffd41d0a43","Type":"ContainerStarted","Data":"09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6"}
Jul 26 15:59:44.341942 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:44.341577    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk" event={"ID":"54b34042-382b-47e0-a65b-4fffd41d0a43","Type":"ContainerStarted","Data":"fb7420786b70b61f721d59f0c187801633679ae96a4809eaa2bc4fa69704c3d2"}
Jul 26 15:59:44.357920 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:44.357857    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk" podStartSLOduration=1.35784263 podStartE2EDuration="1.35784263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:59:44.355790601 +0000 UTC m=+1299.855180439" watchObservedRunningTime="2026-07-26 15:59:44.35784263 +0000 UTC m=+1299.857232467"
Jul 26 15:59:53.478095 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:53.478058    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:53.478095 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:53.478099    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:53.490327 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:53.490300    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 15:59:54.394338 ip-10-0-132-10 kubenswrapper[2560]: I0726 15:59:54.394310    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 16:00:10.070149 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:10.070105    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"]
Jul 26 16:00:25.089998 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.089923    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk" podUID="54b34042-382b-47e0-a65b-4fffd41d0a43" containerName="main" containerID="cri-o://09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6" gracePeriod=45
Jul 26 16:00:25.343904 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.343828    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 16:00:25.373532 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.373510    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54b34042-382b-47e0-a65b-4fffd41d0a43-tls-certs\") pod \"54b34042-382b-47e0-a65b-4fffd41d0a43\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") "
Jul 26 16:00:25.373640 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.373547    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-dshm\") pod \"54b34042-382b-47e0-a65b-4fffd41d0a43\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") "
Jul 26 16:00:25.373640 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.373568    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pq5m\" (UniqueName: \"kubernetes.io/projected/54b34042-382b-47e0-a65b-4fffd41d0a43-kube-api-access-5pq5m\") pod \"54b34042-382b-47e0-a65b-4fffd41d0a43\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") "
Jul 26 16:00:25.373640 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.373629    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-model-cache\") pod \"54b34042-382b-47e0-a65b-4fffd41d0a43\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") "
Jul 26 16:00:25.373785 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.373680    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-home\") pod \"54b34042-382b-47e0-a65b-4fffd41d0a43\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") "
Jul 26 16:00:25.373785 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.373708    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-tmp-dir\") pod \"54b34042-382b-47e0-a65b-4fffd41d0a43\" (UID: \"54b34042-382b-47e0-a65b-4fffd41d0a43\") "
Jul 26 16:00:25.373956 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.373933    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-model-cache" (OuterVolumeSpecName: "model-cache") pod "54b34042-382b-47e0-a65b-4fffd41d0a43" (UID: "54b34042-382b-47e0-a65b-4fffd41d0a43"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:00:25.374043 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.373983    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-home" (OuterVolumeSpecName: "home") pod "54b34042-382b-47e0-a65b-4fffd41d0a43" (UID: "54b34042-382b-47e0-a65b-4fffd41d0a43"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:00:25.374122 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.374104    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "54b34042-382b-47e0-a65b-4fffd41d0a43" (UID: "54b34042-382b-47e0-a65b-4fffd41d0a43"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:00:25.375726 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.375666    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-dshm" (OuterVolumeSpecName: "dshm") pod "54b34042-382b-47e0-a65b-4fffd41d0a43" (UID: "54b34042-382b-47e0-a65b-4fffd41d0a43"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:00:25.375828 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.375780    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54b34042-382b-47e0-a65b-4fffd41d0a43-kube-api-access-5pq5m" (OuterVolumeSpecName: "kube-api-access-5pq5m") pod "54b34042-382b-47e0-a65b-4fffd41d0a43" (UID: "54b34042-382b-47e0-a65b-4fffd41d0a43"). InnerVolumeSpecName "kube-api-access-5pq5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:00:25.375881 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.375864    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54b34042-382b-47e0-a65b-4fffd41d0a43-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "54b34042-382b-47e0-a65b-4fffd41d0a43" (UID: "54b34042-382b-47e0-a65b-4fffd41d0a43"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:00:25.474952 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.474927    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54b34042-382b-47e0-a65b-4fffd41d0a43-tls-certs\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:00:25.474952 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.474950    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-dshm\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:00:25.475091 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.474964    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pq5m\" (UniqueName: \"kubernetes.io/projected/54b34042-382b-47e0-a65b-4fffd41d0a43-kube-api-access-5pq5m\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:00:25.475091 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.474978    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-model-cache\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:00:25.475091 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.474989    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-home\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:00:25.475091 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.475024    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54b34042-382b-47e0-a65b-4fffd41d0a43-tmp-dir\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:00:25.512229 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.512198    2560 generic.go:356] "Generic (PLEG): container finished" podID="54b34042-382b-47e0-a65b-4fffd41d0a43" containerID="09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6" exitCode=0
Jul 26 16:00:25.512317 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.512244    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"
Jul 26 16:00:25.512317 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.512286    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk" event={"ID":"54b34042-382b-47e0-a65b-4fffd41d0a43","Type":"ContainerDied","Data":"09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6"}
Jul 26 16:00:25.512423 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.512326    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk" event={"ID":"54b34042-382b-47e0-a65b-4fffd41d0a43","Type":"ContainerDied","Data":"fb7420786b70b61f721d59f0c187801633679ae96a4809eaa2bc4fa69704c3d2"}
Jul 26 16:00:25.512423 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.512344    2560 scope.go:117] "RemoveContainer" containerID="09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6"
Jul 26 16:00:25.521132 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.521114    2560 scope.go:117] "RemoveContainer" containerID="09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6"
Jul 26 16:00:25.521362 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:00:25.521344    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6\": container with ID starting with 09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6 not found: ID does not exist" containerID="09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6"
Jul 26 16:00:25.521441 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.521369    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6"} err="failed to get container status \"09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6\": rpc error: code = NotFound desc = could not find container \"09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6\": container with ID starting with 09d83125e3389341949b6193a961db4bca956f70e178e65be4948061ec1266a6 not found: ID does not exist"
Jul 26 16:00:25.532679 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.532656    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"]
Jul 26 16:00:25.536074 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:25.536054    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-25ae8ef7-kserve-5c7b86ddf5-sb8nk"]
Jul 26 16:00:27.138831 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:00:27.138799    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54b34042-382b-47e0-a65b-4fffd41d0a43" path="/var/lib/kubelet/pods/54b34042-382b-47e0-a65b-4fffd41d0a43/volumes"
Jul 26 16:03:52.794275 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:52.794193    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"]
Jul 26 16:03:52.794842 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:52.794752    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54b34042-382b-47e0-a65b-4fffd41d0a43" containerName="main"
Jul 26 16:03:52.794842 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:52.794774    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="54b34042-382b-47e0-a65b-4fffd41d0a43" containerName="main"
Jul 26 16:03:52.794984 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:52.794869    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="54b34042-382b-47e0-a65b-4fffd41d0a43" containerName="main"
Jul 26 16:03:52.797576 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:52.797555    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:52.799690 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:52.799655    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2b3f0867-serving-cert\""
Jul 26 16:03:52.806993 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:52.806970    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"]
Jul 26 16:03:52.912466 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:52.912429    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d811fb60-71d3-44b4-85d3-f4c18608e8c7-maas-api-tls\") pod \"maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:52.912614 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:52.912476    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d811fb60-71d3-44b4-85d3-f4c18608e8c7-openshift-service-ca\") pod \"maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:52.912614 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:52.912513    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxrjz\" (UniqueName: \"kubernetes.io/projected/d811fb60-71d3-44b4-85d3-f4c18608e8c7-kube-api-access-kxrjz\") pod \"maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:53.013658 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.013620    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d811fb60-71d3-44b4-85d3-f4c18608e8c7-maas-api-tls\") pod \"maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:53.013869 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.013669    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d811fb60-71d3-44b4-85d3-f4c18608e8c7-openshift-service-ca\") pod \"maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:53.013869 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.013849    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxrjz\" (UniqueName: \"kubernetes.io/projected/d811fb60-71d3-44b4-85d3-f4c18608e8c7-kube-api-access-kxrjz\") pod \"maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:53.014378 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.014355    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d811fb60-71d3-44b4-85d3-f4c18608e8c7-openshift-service-ca\") pod \"maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:53.016084 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.016064    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d811fb60-71d3-44b4-85d3-f4c18608e8c7-maas-api-tls\") pod \"maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:53.021772 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.021748    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxrjz\" (UniqueName: \"kubernetes.io/projected/d811fb60-71d3-44b4-85d3-f4c18608e8c7-kube-api-access-kxrjz\") pod \"maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:53.109623 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.109539    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:53.238835 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.238812    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"]
Jul 26 16:03:53.241037 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:03:53.241012    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd811fb60_71d3_44b4_85d3_f4c18608e8c7.slice/crio-6686dfb55b69f54da99c8199824f70711b8c0b4bc714220736622da39fa0cf0c WatchSource:0}: Error finding container 6686dfb55b69f54da99c8199824f70711b8c0b4bc714220736622da39fa0cf0c: Status 404 returned error can't find the container with id 6686dfb55b69f54da99c8199824f70711b8c0b4bc714220736622da39fa0cf0c
Jul 26 16:03:53.242426 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.242408    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:03:53.390880 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.390850    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq" event={"ID":"d811fb60-71d3-44b4-85d3-f4c18608e8c7","Type":"ContainerStarted","Data":"6686dfb55b69f54da99c8199824f70711b8c0b4bc714220736622da39fa0cf0c"}
Jul 26 16:03:53.405757 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.405731    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"]
Jul 26 16:03:53.410178 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.410154    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:03:53.412752 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.412729    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2b3f0867\""
Jul 26 16:03:53.420409 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.420385    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"]
Jul 26 16:03:53.463030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.463008    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"]
Jul 26 16:03:53.466223 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.466208    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:03:53.468276 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.468255    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-2b3f0867-dockercfg-b29mq\""
Jul 26 16:03:53.477157 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.477135    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"]
Jul 26 16:03:53.518050 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.518021    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5vdd\" (UniqueName: \"kubernetes.io/projected/a52ef70a-08f1-45d6-90ac-a3f1089e835f-kube-api-access-s5vdd\") pod \"payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk\" (UID: \"a52ef70a-08f1-45d6-90ac-a3f1089e835f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:03:53.518226 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.518060    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a52ef70a-08f1-45d6-90ac-a3f1089e835f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk\" (UID: \"a52ef70a-08f1-45d6-90ac-a3f1089e835f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:03:53.618492 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.618461    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5vdd\" (UniqueName: \"kubernetes.io/projected/a52ef70a-08f1-45d6-90ac-a3f1089e835f-kube-api-access-s5vdd\") pod \"payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk\" (UID: \"a52ef70a-08f1-45d6-90ac-a3f1089e835f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:03:53.618663 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.618510    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-plugins-config-volume\") pod \"payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p\" (UID: \"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d\") " pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:03:53.618663 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.618545    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a52ef70a-08f1-45d6-90ac-a3f1089e835f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk\" (UID: \"a52ef70a-08f1-45d6-90ac-a3f1089e835f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:03:53.618663 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.618573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnbxj\" (UniqueName: \"kubernetes.io/projected/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-kube-api-access-hnbxj\") pod \"payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p\" (UID: \"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d\") " pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:03:53.619261 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.619240    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a52ef70a-08f1-45d6-90ac-a3f1089e835f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk\" (UID: \"a52ef70a-08f1-45d6-90ac-a3f1089e835f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:03:53.625919 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.625883    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5vdd\" (UniqueName: \"kubernetes.io/projected/a52ef70a-08f1-45d6-90ac-a3f1089e835f-kube-api-access-s5vdd\") pod \"payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk\" (UID: \"a52ef70a-08f1-45d6-90ac-a3f1089e835f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:03:53.719217 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.719146    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-plugins-config-volume\") pod \"payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p\" (UID: \"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d\") " pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:03:53.719217 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.719186    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnbxj\" (UniqueName: \"kubernetes.io/projected/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-kube-api-access-hnbxj\") pod \"payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p\" (UID: \"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d\") " pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:03:53.719833 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.719813    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-plugins-config-volume\") pod \"payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p\" (UID: \"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d\") " pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:03:53.723049 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.723026    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:03:53.730257 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.730233    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnbxj\" (UniqueName: \"kubernetes.io/projected/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-kube-api-access-hnbxj\") pod \"payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p\" (UID: \"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d\") " pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:03:53.776640 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.776610    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:03:53.863702 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.863678    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"]
Jul 26 16:03:53.867068 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:03:53.867029    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52ef70a_08f1_45d6_90ac_a3f1089e835f.slice/crio-5493c895812281c6a4101f16ec9072e423b96f2f34a08db759acf592a04c018d WatchSource:0}: Error finding container 5493c895812281c6a4101f16ec9072e423b96f2f34a08db759acf592a04c018d: Status 404 returned error can't find the container with id 5493c895812281c6a4101f16ec9072e423b96f2f34a08db759acf592a04c018d
Jul 26 16:03:53.924972 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:53.924882    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"]
Jul 26 16:03:53.925606 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:03:53.925581    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee5921c_61c6_4cf2_a44f_4e3e237f6a0d.slice/crio-196b7b922099cbf80a3f2ee9a80855dafe310dd5925fef10227a795b116edddb WatchSource:0}: Error finding container 196b7b922099cbf80a3f2ee9a80855dafe310dd5925fef10227a795b116edddb: Status 404 returned error can't find the container with id 196b7b922099cbf80a3f2ee9a80855dafe310dd5925fef10227a795b116edddb
Jul 26 16:03:54.396574 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.396542    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq" event={"ID":"d811fb60-71d3-44b4-85d3-f4c18608e8c7","Type":"ContainerStarted","Data":"09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256"}
Jul 26 16:03:54.396757 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.396614    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:03:54.398156 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.398130    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk" event={"ID":"a52ef70a-08f1-45d6-90ac-a3f1089e835f","Type":"ContainerStarted","Data":"dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a"}
Jul 26 16:03:54.398270 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.398182    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:03:54.398270 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.398208    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk" event={"ID":"a52ef70a-08f1-45d6-90ac-a3f1089e835f","Type":"ContainerStarted","Data":"5493c895812281c6a4101f16ec9072e423b96f2f34a08db759acf592a04c018d"}
Jul 26 16:03:54.399539 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.399519    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p" event={"ID":"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d","Type":"ContainerStarted","Data":"1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23"}
Jul 26 16:03:54.399644 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.399554    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p" event={"ID":"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d","Type":"ContainerStarted","Data":"196b7b922099cbf80a3f2ee9a80855dafe310dd5925fef10227a795b116edddb"}
Jul 26 16:03:54.399644 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.399627    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:03:54.413878 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.413825    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq" podStartSLOduration=1.835011121 podStartE2EDuration="2.413806832s" totalImagesPullingTime="578.795711ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:03:54.41292268 +0000 UTC m=+1549.912312523" watchObservedRunningTime="2026-07-26 16:03:54.413806832 +0000 UTC m=+1549.913196671"
Jul 26 16:03:54.428174 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.428128    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk" podStartSLOduration=1.428113126 podStartE2EDuration="1.428113126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:03:54.427299925 +0000 UTC m=+1549.926689764" watchObservedRunningTime="2026-07-26 16:03:54.428113126 +0000 UTC m=+1549.927502963"
Jul 26 16:03:54.442864 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:03:54.442819    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p" podStartSLOduration=1.44280396 podStartE2EDuration="1.44280396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:03:54.44173081 +0000 UTC m=+1549.941120648" watchObservedRunningTime="2026-07-26 16:03:54.44280396 +0000 UTC m=+1549.942193798"
Jul 26 16:04:00.409370 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:00.409341    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:04:05.406524 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:05.406492    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:04:05.406983 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:05.406569    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:04:18.153047 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:18.153005    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"]
Jul 26 16:04:18.156799 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:18.156778    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"
Jul 26 16:04:18.158959 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:18.158939    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:04:18.162935 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:18.162905    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"]
Jul 26 16:04:18.206573 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:18.206540    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f893f7ac-9950-4326-8453-2f840cac781d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm\" (UID: \"f893f7ac-9950-4326-8453-2f840cac781d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"
Jul 26 16:04:18.307538 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:18.307497    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f893f7ac-9950-4326-8453-2f840cac781d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm\" (UID: \"f893f7ac-9950-4326-8453-2f840cac781d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"
Jul 26 16:04:18.308251 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:18.308230    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f893f7ac-9950-4326-8453-2f840cac781d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm\" (UID: \"f893f7ac-9950-4326-8453-2f840cac781d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"
Jul 26 16:04:18.469132 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:18.469038    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"
Jul 26 16:04:18.604385 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:18.604359    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"]
Jul 26 16:04:18.606665 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:04:18.606635    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf893f7ac_9950_4326_8453_2f840cac781d.slice/crio-977584af79669ee3333f23b37e22a1f910cf2df49f6b3d9240386bf4b226ece9 WatchSource:0}: Error finding container 977584af79669ee3333f23b37e22a1f910cf2df49f6b3d9240386bf4b226ece9: Status 404 returned error can't find the container with id 977584af79669ee3333f23b37e22a1f910cf2df49f6b3d9240386bf4b226ece9
Jul 26 16:04:19.509734 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:19.509693    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm" event={"ID":"f893f7ac-9950-4326-8453-2f840cac781d","Type":"ContainerStarted","Data":"977584af79669ee3333f23b37e22a1f910cf2df49f6b3d9240386bf4b226ece9"}
Jul 26 16:04:21.522368 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:21.522327    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm" event={"ID":"f893f7ac-9950-4326-8453-2f840cac781d","Type":"ContainerStarted","Data":"ad567771b2137c7720bbe450d52e50f2f5babb36fbd12fce780e4b87a73c4205"}
Jul 26 16:04:21.537814 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:21.537758    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm" podStartSLOduration=1.05092716 podStartE2EDuration="3.5377407s" totalImagesPullingTime="2.48681354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:04:21.536019511 +0000 UTC m=+1577.035409376" watchObservedRunningTime="2026-07-26 16:04:21.5377407 +0000 UTC m=+1577.037130538"
Jul 26 16:04:22.528858 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:22.528826    2560 generic.go:356] "Generic (PLEG): container finished" podID="f893f7ac-9950-4326-8453-2f840cac781d" containerID="ad567771b2137c7720bbe450d52e50f2f5babb36fbd12fce780e4b87a73c4205" exitCode=0
Jul 26 16:04:22.529259 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:22.528919    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm" event={"ID":"f893f7ac-9950-4326-8453-2f840cac781d","Type":"ContainerDied","Data":"ad567771b2137c7720bbe450d52e50f2f5babb36fbd12fce780e4b87a73c4205"}
Jul 26 16:04:23.662465 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:23.662439    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"
Jul 26 16:04:23.755871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:23.755839    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f893f7ac-9950-4326-8453-2f840cac781d-maas-api-service-ca\") pod \"f893f7ac-9950-4326-8453-2f840cac781d\" (UID: \"f893f7ac-9950-4326-8453-2f840cac781d\") "
Jul 26 16:04:23.756218 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:23.756193    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f893f7ac-9950-4326-8453-2f840cac781d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f893f7ac-9950-4326-8453-2f840cac781d" (UID: "f893f7ac-9950-4326-8453-2f840cac781d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:04:23.856535 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:23.856465    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f893f7ac-9950-4326-8453-2f840cac781d-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:04:24.538527 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:24.538499    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"
Jul 26 16:04:24.538527 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:24.538514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm" event={"ID":"f893f7ac-9950-4326-8453-2f840cac781d","Type":"ContainerDied","Data":"977584af79669ee3333f23b37e22a1f910cf2df49f6b3d9240386bf4b226ece9"}
Jul 26 16:04:24.538730 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:24.538557    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="977584af79669ee3333f23b37e22a1f910cf2df49f6b3d9240386bf4b226ece9"
Jul 26 16:04:28.558276 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:28.558242    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"]
Jul 26 16:04:28.559077 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:28.559021    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq" podUID="d811fb60-71d3-44b4-85d3-f4c18608e8c7" containerName="maas-api" containerID="cri-o://09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256" gracePeriod=30
Jul 26 16:04:28.615619 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:28.615580    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"]
Jul 26 16:04:28.615928 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:28.615876    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p" podUID="0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d" containerName="payload-processing" containerID="cri-o://1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23" gracePeriod=30
Jul 26 16:04:28.621548 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:28.621517    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"]
Jul 26 16:04:28.621784 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:28.621737    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk" podUID="a52ef70a-08f1-45d6-90ac-a3f1089e835f" containerName="payload-pre-processing" containerID="cri-o://dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a" gracePeriod=30
Jul 26 16:04:28.879470 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:28.879443    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:04:28.929180 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:28.929155    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:04:28.932220 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:28.932200    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:04:29.007717 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.007686    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d811fb60-71d3-44b4-85d3-f4c18608e8c7-openshift-service-ca\") pod \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") "
Jul 26 16:04:29.007907 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.007735    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxrjz\" (UniqueName: \"kubernetes.io/projected/d811fb60-71d3-44b4-85d3-f4c18608e8c7-kube-api-access-kxrjz\") pod \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") "
Jul 26 16:04:29.007907 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.007770    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a52ef70a-08f1-45d6-90ac-a3f1089e835f-plugins-config-volume\") pod \"a52ef70a-08f1-45d6-90ac-a3f1089e835f\" (UID: \"a52ef70a-08f1-45d6-90ac-a3f1089e835f\") "
Jul 26 16:04:29.007907 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.007809    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5vdd\" (UniqueName: \"kubernetes.io/projected/a52ef70a-08f1-45d6-90ac-a3f1089e835f-kube-api-access-s5vdd\") pod \"a52ef70a-08f1-45d6-90ac-a3f1089e835f\" (UID: \"a52ef70a-08f1-45d6-90ac-a3f1089e835f\") "
Jul 26 16:04:29.007907 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.007838    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-plugins-config-volume\") pod \"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d\" (UID: \"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d\") "
Jul 26 16:04:29.008126 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.007926    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d811fb60-71d3-44b4-85d3-f4c18608e8c7-maas-api-tls\") pod \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\" (UID: \"d811fb60-71d3-44b4-85d3-f4c18608e8c7\") "
Jul 26 16:04:29.008126 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.007952    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnbxj\" (UniqueName: \"kubernetes.io/projected/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-kube-api-access-hnbxj\") pod \"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d\" (UID: \"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d\") "
Jul 26 16:04:29.008224 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.008115    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d811fb60-71d3-44b4-85d3-f4c18608e8c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d811fb60-71d3-44b4-85d3-f4c18608e8c7" (UID: "d811fb60-71d3-44b4-85d3-f4c18608e8c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:04:29.008224 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.008181    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52ef70a-08f1-45d6-90ac-a3f1089e835f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a52ef70a-08f1-45d6-90ac-a3f1089e835f" (UID: "a52ef70a-08f1-45d6-90ac-a3f1089e835f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:04:29.008395 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.008237    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d811fb60-71d3-44b4-85d3-f4c18608e8c7-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:04:29.008395 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.008259    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a52ef70a-08f1-45d6-90ac-a3f1089e835f-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:04:29.008590 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.008558    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d" (UID: "0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:04:29.010080 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.010046    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52ef70a-08f1-45d6-90ac-a3f1089e835f-kube-api-access-s5vdd" (OuterVolumeSpecName: "kube-api-access-s5vdd") pod "a52ef70a-08f1-45d6-90ac-a3f1089e835f" (UID: "a52ef70a-08f1-45d6-90ac-a3f1089e835f"). InnerVolumeSpecName "kube-api-access-s5vdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:04:29.010633 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.010604    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-kube-api-access-hnbxj" (OuterVolumeSpecName: "kube-api-access-hnbxj") pod "0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d" (UID: "0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d"). InnerVolumeSpecName "kube-api-access-hnbxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:04:29.010717 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.010641    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d811fb60-71d3-44b4-85d3-f4c18608e8c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d811fb60-71d3-44b4-85d3-f4c18608e8c7" (UID: "d811fb60-71d3-44b4-85d3-f4c18608e8c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:04:29.010989 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.010962    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d811fb60-71d3-44b4-85d3-f4c18608e8c7-kube-api-access-kxrjz" (OuterVolumeSpecName: "kube-api-access-kxrjz") pod "d811fb60-71d3-44b4-85d3-f4c18608e8c7" (UID: "d811fb60-71d3-44b4-85d3-f4c18608e8c7"). InnerVolumeSpecName "kube-api-access-kxrjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:04:29.108721 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.108636    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5vdd\" (UniqueName: \"kubernetes.io/projected/a52ef70a-08f1-45d6-90ac-a3f1089e835f-kube-api-access-s5vdd\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:04:29.108721 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.108668    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:04:29.108721 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.108682    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d811fb60-71d3-44b4-85d3-f4c18608e8c7-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:04:29.108721 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.108694    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnbxj\" (UniqueName: \"kubernetes.io/projected/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d-kube-api-access-hnbxj\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:04:29.108721 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.108705    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxrjz\" (UniqueName: \"kubernetes.io/projected/d811fb60-71d3-44b4-85d3-f4c18608e8c7-kube-api-access-kxrjz\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:04:29.564874 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.564840    2560 generic.go:356] "Generic (PLEG): container finished" podID="a52ef70a-08f1-45d6-90ac-a3f1089e835f" containerID="dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a" exitCode=0
Jul 26 16:04:29.565339 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.564900    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"
Jul 26 16:04:29.565339 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.564926    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk" event={"ID":"a52ef70a-08f1-45d6-90ac-a3f1089e835f","Type":"ContainerDied","Data":"dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a"}
Jul 26 16:04:29.565339 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.564976    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk" event={"ID":"a52ef70a-08f1-45d6-90ac-a3f1089e835f","Type":"ContainerDied","Data":"5493c895812281c6a4101f16ec9072e423b96f2f34a08db759acf592a04c018d"}
Jul 26 16:04:29.565339 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.565010    2560 scope.go:117] "RemoveContainer" containerID="dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a"
Jul 26 16:04:29.566363 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.566344    2560 generic.go:356] "Generic (PLEG): container finished" podID="0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d" containerID="1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23" exitCode=0
Jul 26 16:04:29.566425 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.566386    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"
Jul 26 16:04:29.566483 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.566434    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p" event={"ID":"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d","Type":"ContainerDied","Data":"1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23"}
Jul 26 16:04:29.566538 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.566486    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p" event={"ID":"0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d","Type":"ContainerDied","Data":"196b7b922099cbf80a3f2ee9a80855dafe310dd5925fef10227a795b116edddb"}
Jul 26 16:04:29.567812 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.567792    2560 generic.go:356] "Generic (PLEG): container finished" podID="d811fb60-71d3-44b4-85d3-f4c18608e8c7" containerID="09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256" exitCode=0
Jul 26 16:04:29.567924 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.567824    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq" event={"ID":"d811fb60-71d3-44b4-85d3-f4c18608e8c7","Type":"ContainerDied","Data":"09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256"}
Jul 26 16:04:29.567924 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.567834    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"
Jul 26 16:04:29.567924 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.567861    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq" event={"ID":"d811fb60-71d3-44b4-85d3-f4c18608e8c7","Type":"ContainerDied","Data":"6686dfb55b69f54da99c8199824f70711b8c0b4bc714220736622da39fa0cf0c"}
Jul 26 16:04:29.585727 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.585708    2560 scope.go:117] "RemoveContainer" containerID="dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a"
Jul 26 16:04:29.586053 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:04:29.586026    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a\": container with ID starting with dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a not found: ID does not exist" containerID="dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a"
Jul 26 16:04:29.586152 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.586061    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a"} err="failed to get container status \"dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a\": rpc error: code = NotFound desc = could not find container \"dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a\": container with ID starting with dab5e014ae8eb90f649ef546f0e67854c14c8b3554e98b9961d3df21f2639e5a not found: ID does not exist"
Jul 26 16:04:29.586152 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.586078    2560 scope.go:117] "RemoveContainer" containerID="1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23"
Jul 26 16:04:29.594076 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.594049    2560 scope.go:117] "RemoveContainer" containerID="1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23"
Jul 26 16:04:29.594245 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.594220    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"]
Jul 26 16:04:29.594342 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:04:29.594313    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23\": container with ID starting with 1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23 not found: ID does not exist" containerID="1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23"
Jul 26 16:04:29.594406 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.594339    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23"} err="failed to get container status \"1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23\": rpc error: code = NotFound desc = could not find container \"1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23\": container with ID starting with 1d5fc816581c3511a42461f6c06dab6e76dc48b4797030831f4bff12de184c23 not found: ID does not exist"
Jul 26 16:04:29.594406 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.594357    2560 scope.go:117] "RemoveContainer" containerID="09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256"
Jul 26 16:04:29.597659 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.597636    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b3f0867-6fdc9667f6-s9fjk"]
Jul 26 16:04:29.602089 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.602072    2560 scope.go:117] "RemoveContainer" containerID="09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256"
Jul 26 16:04:29.602343 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:04:29.602323    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256\": container with ID starting with 09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256 not found: ID does not exist" containerID="09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256"
Jul 26 16:04:29.602408 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.602349    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256"} err="failed to get container status \"09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256\": rpc error: code = NotFound desc = could not find container \"09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256\": container with ID starting with 09bf30a35ba552de92deb62618226b13e886f09d48ae895a46da94fa60e53256 not found: ID does not exist"
Jul 26 16:04:29.607832 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.607812    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"]
Jul 26 16:04:29.612145 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.612124    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2b3f0867-54c4bc47fb-bzq2p"]
Jul 26 16:04:29.623753 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.623723    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"]
Jul 26 16:04:29.629358 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:29.629314    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2b3f0867-6d4b6cf855-twjnq"]
Jul 26 16:04:31.138140 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:31.138103    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d" path="/var/lib/kubelet/pods/0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d/volumes"
Jul 26 16:04:31.138615 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:31.138459    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a52ef70a-08f1-45d6-90ac-a3f1089e835f" path="/var/lib/kubelet/pods/a52ef70a-08f1-45d6-90ac-a3f1089e835f/volumes"
Jul 26 16:04:31.138790 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:31.138778    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d811fb60-71d3-44b4-85d3-f4c18608e8c7" path="/var/lib/kubelet/pods/d811fb60-71d3-44b4-85d3-f4c18608e8c7/volumes"
Jul 26 16:04:33.233770 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:33.233734    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"]
Jul 26 16:04:33.236306 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:33.236281    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2b3f0867-xsjrm"]
Jul 26 16:04:35.139018 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:35.138987    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f893f7ac-9950-4326-8453-2f840cac781d" path="/var/lib/kubelet/pods/f893f7ac-9950-4326-8453-2f840cac781d/volumes"
Jul 26 16:04:37.099679 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.099640    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"]
Jul 26 16:04:37.100199 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100181    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f893f7ac-9950-4326-8453-2f840cac781d" containerName="revoke-keys"
Jul 26 16:04:37.100273 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100202    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f893f7ac-9950-4326-8453-2f840cac781d" containerName="revoke-keys"
Jul 26 16:04:37.100273 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100225    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d811fb60-71d3-44b4-85d3-f4c18608e8c7" containerName="maas-api"
Jul 26 16:04:37.100273 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100235    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d811fb60-71d3-44b4-85d3-f4c18608e8c7" containerName="maas-api"
Jul 26 16:04:37.100273 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100248    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a52ef70a-08f1-45d6-90ac-a3f1089e835f" containerName="payload-pre-processing"
Jul 26 16:04:37.100273 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100258    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a52ef70a-08f1-45d6-90ac-a3f1089e835f" containerName="payload-pre-processing"
Jul 26 16:04:37.100273 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100272    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d" containerName="payload-processing"
Jul 26 16:04:37.100555 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100280    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d" containerName="payload-processing"
Jul 26 16:04:37.100555 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100362    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d811fb60-71d3-44b4-85d3-f4c18608e8c7" containerName="maas-api"
Jul 26 16:04:37.100555 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100375    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ee5921c-61c6-4cf2-a44f-4e3e237f6a0d" containerName="payload-processing"
Jul 26 16:04:37.100555 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100386    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a52ef70a-08f1-45d6-90ac-a3f1089e835f" containerName="payload-pre-processing"
Jul 26 16:04:37.100555 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.100398    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f893f7ac-9950-4326-8453-2f840cac781d" containerName="revoke-keys"
Jul 26 16:04:37.102823 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.102801    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:37.104824 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.104799    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-28d9404a-serving-cert\""
Jul 26 16:04:37.113246 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.113222    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"]
Jul 26 16:04:37.183958 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.178294    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22f9f5f-a559-45e1-9920-fceadde8ee76-openshift-service-ca\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:37.183958 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.178390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:37.183958 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.178516    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5kcl\" (UniqueName: \"kubernetes.io/projected/e22f9f5f-a559-45e1-9920-fceadde8ee76-kube-api-access-n5kcl\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:37.279262 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.279224    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kcl\" (UniqueName: \"kubernetes.io/projected/e22f9f5f-a559-45e1-9920-fceadde8ee76-kube-api-access-n5kcl\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:37.279438 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.279284    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22f9f5f-a559-45e1-9920-fceadde8ee76-openshift-service-ca\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:37.279438 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.279321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:37.279538 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:04:37.279443    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-serving-cert: secret "maas-api-e2e-migrate-28d9404a-serving-cert" not found
Jul 26 16:04:37.279538 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:04:37.279520    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls podName:e22f9f5f-a559-45e1-9920-fceadde8ee76 nodeName:}" failed. No retries permitted until 2026-07-26 16:04:37.779497168 +0000 UTC m=+1593.278886987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls") pod "maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg" (UID: "e22f9f5f-a559-45e1-9920-fceadde8ee76") : secret "maas-api-e2e-migrate-28d9404a-serving-cert" not found
Jul 26 16:04:37.280071 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.280051    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22f9f5f-a559-45e1-9920-fceadde8ee76-openshift-service-ca\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:37.292660 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.292632    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5kcl\" (UniqueName: \"kubernetes.io/projected/e22f9f5f-a559-45e1-9920-fceadde8ee76-kube-api-access-n5kcl\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:37.741955 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.741918    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"]
Jul 26 16:04:37.744812 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.744795    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:04:37.747329 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.747305    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-28d9404a-dockercfg-rnzfl\""
Jul 26 16:04:37.747436 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.747305    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-28d9404a\""
Jul 26 16:04:37.757224 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.757199    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"]
Jul 26 16:04:37.772253 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.772223    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"]
Jul 26 16:04:37.775121 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.775100    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:04:37.784311 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.784289    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:37.784458 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:04:37.784442    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-serving-cert: secret "maas-api-e2e-migrate-28d9404a-serving-cert" not found
Jul 26 16:04:37.784563 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:04:37.784554    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls podName:e22f9f5f-a559-45e1-9920-fceadde8ee76 nodeName:}" failed. No retries permitted until 2026-07-26 16:04:38.784538889 +0000 UTC m=+1594.283928705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls") pod "maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg" (UID: "e22f9f5f-a559-45e1-9920-fceadde8ee76") : secret "maas-api-e2e-migrate-28d9404a-serving-cert" not found
Jul 26 16:04:37.787183 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.787155    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"]
Jul 26 16:04:37.885105 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.885066    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7643acb1-46e6-4322-a31c-e46b9ce4a06d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84\" (UID: \"7643acb1-46e6-4322-a31c-e46b9ce4a06d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:04:37.885269 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.885127    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbzhl\" (UniqueName: \"kubernetes.io/projected/7643acb1-46e6-4322-a31c-e46b9ce4a06d-kube-api-access-tbzhl\") pod \"payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84\" (UID: \"7643acb1-46e6-4322-a31c-e46b9ce4a06d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:04:37.885269 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.885233    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5be5e0ea-f413-493f-a2b2-4ee588a37943-plugins-config-volume\") pod \"payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb\" (UID: \"5be5e0ea-f413-493f-a2b2-4ee588a37943\") " pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:04:37.885347 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.885278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmnzn\" (UniqueName: \"kubernetes.io/projected/5be5e0ea-f413-493f-a2b2-4ee588a37943-kube-api-access-cmnzn\") pod \"payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb\" (UID: \"5be5e0ea-f413-493f-a2b2-4ee588a37943\") " pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:04:37.985995 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.985957    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5be5e0ea-f413-493f-a2b2-4ee588a37943-plugins-config-volume\") pod \"payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb\" (UID: \"5be5e0ea-f413-493f-a2b2-4ee588a37943\") " pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:04:37.986174 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.986004    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmnzn\" (UniqueName: \"kubernetes.io/projected/5be5e0ea-f413-493f-a2b2-4ee588a37943-kube-api-access-cmnzn\") pod \"payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb\" (UID: \"5be5e0ea-f413-493f-a2b2-4ee588a37943\") " pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:04:37.986174 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.986039    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7643acb1-46e6-4322-a31c-e46b9ce4a06d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84\" (UID: \"7643acb1-46e6-4322-a31c-e46b9ce4a06d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:04:37.986174 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.986065    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbzhl\" (UniqueName: \"kubernetes.io/projected/7643acb1-46e6-4322-a31c-e46b9ce4a06d-kube-api-access-tbzhl\") pod \"payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84\" (UID: \"7643acb1-46e6-4322-a31c-e46b9ce4a06d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:04:37.986924 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.986881    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5be5e0ea-f413-493f-a2b2-4ee588a37943-plugins-config-volume\") pod \"payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb\" (UID: \"5be5e0ea-f413-493f-a2b2-4ee588a37943\") " pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:04:37.986994 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.986881    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7643acb1-46e6-4322-a31c-e46b9ce4a06d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84\" (UID: \"7643acb1-46e6-4322-a31c-e46b9ce4a06d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:04:37.994536 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.994486    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmnzn\" (UniqueName: \"kubernetes.io/projected/5be5e0ea-f413-493f-a2b2-4ee588a37943-kube-api-access-cmnzn\") pod \"payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb\" (UID: \"5be5e0ea-f413-493f-a2b2-4ee588a37943\") " pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:04:37.994609 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:37.994551    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbzhl\" (UniqueName: \"kubernetes.io/projected/7643acb1-46e6-4322-a31c-e46b9ce4a06d-kube-api-access-tbzhl\") pod \"payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84\" (UID: \"7643acb1-46e6-4322-a31c-e46b9ce4a06d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:04:38.055696 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.055662    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:04:38.085271 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.085229    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:04:38.204914 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.204872    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"]
Jul 26 16:04:38.207344 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:04:38.207317    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7643acb1_46e6_4322_a31c_e46b9ce4a06d.slice/crio-a8ca307c19af04d7783837ce2ee3192edfaad66ac7b56e665e8188c517f6a0f9 WatchSource:0}: Error finding container a8ca307c19af04d7783837ce2ee3192edfaad66ac7b56e665e8188c517f6a0f9: Status 404 returned error can't find the container with id a8ca307c19af04d7783837ce2ee3192edfaad66ac7b56e665e8188c517f6a0f9
Jul 26 16:04:38.232842 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.232731    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"]
Jul 26 16:04:38.238153 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:04:38.238126    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be5e0ea_f413_493f_a2b2_4ee588a37943.slice/crio-e61f82cf51f2013821a17e940930aa89829c6f0f54462deb7a5a750c2cb417f2 WatchSource:0}: Error finding container e61f82cf51f2013821a17e940930aa89829c6f0f54462deb7a5a750c2cb417f2: Status 404 returned error can't find the container with id e61f82cf51f2013821a17e940930aa89829c6f0f54462deb7a5a750c2cb417f2
Jul 26 16:04:38.611539 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.611438    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84" event={"ID":"7643acb1-46e6-4322-a31c-e46b9ce4a06d","Type":"ContainerStarted","Data":"33a5f52476a0ec29ccd72ae07d7667b60d31e7ca7df7a74fcc68086b58deb135"}
Jul 26 16:04:38.611539 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.611503    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84" event={"ID":"7643acb1-46e6-4322-a31c-e46b9ce4a06d","Type":"ContainerStarted","Data":"a8ca307c19af04d7783837ce2ee3192edfaad66ac7b56e665e8188c517f6a0f9"}
Jul 26 16:04:38.611539 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.611532    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:04:38.613592 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.613539    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb" event={"ID":"5be5e0ea-f413-493f-a2b2-4ee588a37943","Type":"ContainerStarted","Data":"d8fa9f76ba52741c1229260eaf1c32550bc1d5fb111d7f34a115c18e61868044"}
Jul 26 16:04:38.613592 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.613592    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb" event={"ID":"5be5e0ea-f413-493f-a2b2-4ee588a37943","Type":"ContainerStarted","Data":"e61f82cf51f2013821a17e940930aa89829c6f0f54462deb7a5a750c2cb417f2"}
Jul 26 16:04:38.613877 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.613622    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:04:38.626766 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.626712    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84" podStartSLOduration=1.626694721 podStartE2EDuration="1.626694721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:04:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:04:38.625935484 +0000 UTC m=+1594.125325323" watchObservedRunningTime="2026-07-26 16:04:38.626694721 +0000 UTC m=+1594.126084568"
Jul 26 16:04:38.642576 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.642523    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb" podStartSLOduration=1.642504836 podStartE2EDuration="1.642504836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:04:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:04:38.640081915 +0000 UTC m=+1594.139471753" watchObservedRunningTime="2026-07-26 16:04:38.642504836 +0000 UTC m=+1594.141894677"
Jul 26 16:04:38.793343 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.793300    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:38.795715 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.795694    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls\") pod \"maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:38.915722 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:38.915691    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:39.054825 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:39.054796    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"]
Jul 26 16:04:39.056755 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:04:39.056719    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22f9f5f_a559_45e1_9920_fceadde8ee76.slice/crio-19c56af916da0d2f43fdf38a288b8b5c70be838498c20556f89a5a51b42bd93d WatchSource:0}: Error finding container 19c56af916da0d2f43fdf38a288b8b5c70be838498c20556f89a5a51b42bd93d: Status 404 returned error can't find the container with id 19c56af916da0d2f43fdf38a288b8b5c70be838498c20556f89a5a51b42bd93d
Jul 26 16:04:39.624261 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:39.623670    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg" event={"ID":"e22f9f5f-a559-45e1-9920-fceadde8ee76","Type":"ContainerStarted","Data":"fb15718a26546674b62bdd86479a310f4eb41c7617b84e5b13c44e74b90192e3"}
Jul 26 16:04:39.624261 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:39.623730    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg" event={"ID":"e22f9f5f-a559-45e1-9920-fceadde8ee76","Type":"ContainerStarted","Data":"19c56af916da0d2f43fdf38a288b8b5c70be838498c20556f89a5a51b42bd93d"}
Jul 26 16:04:39.624261 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:39.624205    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:39.639959 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:39.639913    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg" podStartSLOduration=2.190970058 podStartE2EDuration="2.639883523s" totalImagesPullingTime="448.913465ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:04:39.638717351 +0000 UTC m=+1595.138107190" watchObservedRunningTime="2026-07-26 16:04:39.639883523 +0000 UTC m=+1595.139273363"
Jul 26 16:04:46.642984 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:46.642952    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:04:49.625242 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:49.625209    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:04:49.625672 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:04:49.625615    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:05:02.010381 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.010341    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"]
Jul 26 16:05:02.016200 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.016179    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"
Jul 26 16:05:02.018565 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.018543    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:05:02.019864 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.019839    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"]
Jul 26 16:05:02.178453 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.178421    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece6f91b-b4b9-42f1-98a9-9d448b1ebd27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc\" (UID: \"ece6f91b-b4b9-42f1-98a9-9d448b1ebd27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"
Jul 26 16:05:02.279526 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.279457    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece6f91b-b4b9-42f1-98a9-9d448b1ebd27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc\" (UID: \"ece6f91b-b4b9-42f1-98a9-9d448b1ebd27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"
Jul 26 16:05:02.280126 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.280106    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece6f91b-b4b9-42f1-98a9-9d448b1ebd27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc\" (UID: \"ece6f91b-b4b9-42f1-98a9-9d448b1ebd27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"
Jul 26 16:05:02.329011 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.328984    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"
Jul 26 16:05:02.462488 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.462465    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"]
Jul 26 16:05:02.465026 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:05:02.464994    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece6f91b_b4b9_42f1_98a9_9d448b1ebd27.slice/crio-3ab4a3b5177c182ac414819e403642df79710bb8a0cbe8f6426e52c72665682c WatchSource:0}: Error finding container 3ab4a3b5177c182ac414819e403642df79710bb8a0cbe8f6426e52c72665682c: Status 404 returned error can't find the container with id 3ab4a3b5177c182ac414819e403642df79710bb8a0cbe8f6426e52c72665682c
Jul 26 16:05:02.735145 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.735115    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc" event={"ID":"ece6f91b-b4b9-42f1-98a9-9d448b1ebd27","Type":"ContainerStarted","Data":"09f79c2d1b2dd993ee78c53c5bce60fe6a1b4d6f20c4e54e456726877327fbab"}
Jul 26 16:05:02.735261 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.735171    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc" event={"ID":"ece6f91b-b4b9-42f1-98a9-9d448b1ebd27","Type":"ContainerStarted","Data":"3ab4a3b5177c182ac414819e403642df79710bb8a0cbe8f6426e52c72665682c"}
Jul 26 16:05:02.747761 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:02.747711    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc" podStartSLOduration=1.747693258 podStartE2EDuration="1.747693258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:05:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:05:02.74737598 +0000 UTC m=+1618.246765830" watchObservedRunningTime="2026-07-26 16:05:02.747693258 +0000 UTC m=+1618.247083099"
Jul 26 16:05:03.741275 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:03.741246    2560 generic.go:356] "Generic (PLEG): container finished" podID="ece6f91b-b4b9-42f1-98a9-9d448b1ebd27" containerID="09f79c2d1b2dd993ee78c53c5bce60fe6a1b4d6f20c4e54e456726877327fbab" exitCode=0
Jul 26 16:05:03.741645 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:03.741300    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc" event={"ID":"ece6f91b-b4b9-42f1-98a9-9d448b1ebd27","Type":"ContainerDied","Data":"09f79c2d1b2dd993ee78c53c5bce60fe6a1b4d6f20c4e54e456726877327fbab"}
Jul 26 16:05:04.878125 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:04.878100    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"
Jul 26 16:05:05.000364 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:05.000302    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece6f91b-b4b9-42f1-98a9-9d448b1ebd27-maas-api-service-ca\") pod \"ece6f91b-b4b9-42f1-98a9-9d448b1ebd27\" (UID: \"ece6f91b-b4b9-42f1-98a9-9d448b1ebd27\") "
Jul 26 16:05:05.000594 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:05.000574    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ece6f91b-b4b9-42f1-98a9-9d448b1ebd27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ece6f91b-b4b9-42f1-98a9-9d448b1ebd27" (UID: "ece6f91b-b4b9-42f1-98a9-9d448b1ebd27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:05:05.101586 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:05.101564    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ece6f91b-b4b9-42f1-98a9-9d448b1ebd27-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:05.751225 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:05.751197    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"
Jul 26 16:05:05.751225 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:05.751206    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc" event={"ID":"ece6f91b-b4b9-42f1-98a9-9d448b1ebd27","Type":"ContainerDied","Data":"3ab4a3b5177c182ac414819e403642df79710bb8a0cbe8f6426e52c72665682c"}
Jul 26 16:05:05.751515 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:05.751247    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ab4a3b5177c182ac414819e403642df79710bb8a0cbe8f6426e52c72665682c"
Jul 26 16:05:12.491349 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.491317    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"]
Jul 26 16:05:12.493886 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.491536    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg" podUID="e22f9f5f-a559-45e1-9920-fceadde8ee76" containerName="maas-api" containerID="cri-o://fb15718a26546674b62bdd86479a310f4eb41c7617b84e5b13c44e74b90192e3" gracePeriod=30
Jul 26 16:05:12.543802 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.543766    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"]
Jul 26 16:05:12.544031 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.544005    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb" podUID="5be5e0ea-f413-493f-a2b2-4ee588a37943" containerName="payload-processing" containerID="cri-o://d8fa9f76ba52741c1229260eaf1c32550bc1d5fb111d7f34a115c18e61868044" gracePeriod=30
Jul 26 16:05:12.548485 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.548458    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"]
Jul 26 16:05:12.548675 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.548654    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84" podUID="7643acb1-46e6-4322-a31c-e46b9ce4a06d" containerName="payload-pre-processing" containerID="cri-o://33a5f52476a0ec29ccd72ae07d7667b60d31e7ca7df7a74fcc68086b58deb135" gracePeriod=30
Jul 26 16:05:12.784338 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.784298    2560 generic.go:356] "Generic (PLEG): container finished" podID="7643acb1-46e6-4322-a31c-e46b9ce4a06d" containerID="33a5f52476a0ec29ccd72ae07d7667b60d31e7ca7df7a74fcc68086b58deb135" exitCode=0
Jul 26 16:05:12.784495 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.784369    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84" event={"ID":"7643acb1-46e6-4322-a31c-e46b9ce4a06d","Type":"ContainerDied","Data":"33a5f52476a0ec29ccd72ae07d7667b60d31e7ca7df7a74fcc68086b58deb135"}
Jul 26 16:05:12.786486 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.786284    2560 generic.go:356] "Generic (PLEG): container finished" podID="e22f9f5f-a559-45e1-9920-fceadde8ee76" containerID="fb15718a26546674b62bdd86479a310f4eb41c7617b84e5b13c44e74b90192e3" exitCode=0
Jul 26 16:05:12.786486 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.786398    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg" event={"ID":"e22f9f5f-a559-45e1-9920-fceadde8ee76","Type":"ContainerDied","Data":"fb15718a26546674b62bdd86479a310f4eb41c7617b84e5b13c44e74b90192e3"}
Jul 26 16:05:12.788609 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.788583    2560 generic.go:356] "Generic (PLEG): container finished" podID="5be5e0ea-f413-493f-a2b2-4ee588a37943" containerID="d8fa9f76ba52741c1229260eaf1c32550bc1d5fb111d7f34a115c18e61868044" exitCode=0
Jul 26 16:05:12.788717 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.788655    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb" event={"ID":"5be5e0ea-f413-493f-a2b2-4ee588a37943","Type":"ContainerDied","Data":"d8fa9f76ba52741c1229260eaf1c32550bc1d5fb111d7f34a115c18e61868044"}
Jul 26 16:05:12.820816 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.820790    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:05:12.823424 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.823398    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:05:12.842521 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.842496    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:05:12.958631 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.958596    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22f9f5f-a559-45e1-9920-fceadde8ee76-openshift-service-ca\") pod \"e22f9f5f-a559-45e1-9920-fceadde8ee76\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") "
Jul 26 16:05:12.958631 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.958632    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmnzn\" (UniqueName: \"kubernetes.io/projected/5be5e0ea-f413-493f-a2b2-4ee588a37943-kube-api-access-cmnzn\") pod \"5be5e0ea-f413-493f-a2b2-4ee588a37943\" (UID: \"5be5e0ea-f413-493f-a2b2-4ee588a37943\") "
Jul 26 16:05:12.958854 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.958676    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls\") pod \"e22f9f5f-a559-45e1-9920-fceadde8ee76\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") "
Jul 26 16:05:12.958854 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.958705    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7643acb1-46e6-4322-a31c-e46b9ce4a06d-plugins-config-volume\") pod \"7643acb1-46e6-4322-a31c-e46b9ce4a06d\" (UID: \"7643acb1-46e6-4322-a31c-e46b9ce4a06d\") "
Jul 26 16:05:12.958854 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.958754    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5be5e0ea-f413-493f-a2b2-4ee588a37943-plugins-config-volume\") pod \"5be5e0ea-f413-493f-a2b2-4ee588a37943\" (UID: \"5be5e0ea-f413-493f-a2b2-4ee588a37943\") "
Jul 26 16:05:12.959043 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.958844    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbzhl\" (UniqueName: \"kubernetes.io/projected/7643acb1-46e6-4322-a31c-e46b9ce4a06d-kube-api-access-tbzhl\") pod \"7643acb1-46e6-4322-a31c-e46b9ce4a06d\" (UID: \"7643acb1-46e6-4322-a31c-e46b9ce4a06d\") "
Jul 26 16:05:12.959043 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.958923    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5kcl\" (UniqueName: \"kubernetes.io/projected/e22f9f5f-a559-45e1-9920-fceadde8ee76-kube-api-access-n5kcl\") pod \"e22f9f5f-a559-45e1-9920-fceadde8ee76\" (UID: \"e22f9f5f-a559-45e1-9920-fceadde8ee76\") "
Jul 26 16:05:12.959140 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.959048    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e22f9f5f-a559-45e1-9920-fceadde8ee76-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e22f9f5f-a559-45e1-9920-fceadde8ee76" (UID: "e22f9f5f-a559-45e1-9920-fceadde8ee76"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:05:12.959200 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.959154    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7643acb1-46e6-4322-a31c-e46b9ce4a06d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7643acb1-46e6-4322-a31c-e46b9ce4a06d" (UID: "7643acb1-46e6-4322-a31c-e46b9ce4a06d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:05:12.959200 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.959153    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5be5e0ea-f413-493f-a2b2-4ee588a37943-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5be5e0ea-f413-493f-a2b2-4ee588a37943" (UID: "5be5e0ea-f413-493f-a2b2-4ee588a37943"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:05:12.959313 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.959246    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7643acb1-46e6-4322-a31c-e46b9ce4a06d-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:12.959313 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.959263    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5be5e0ea-f413-493f-a2b2-4ee588a37943-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:12.959313 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.959273    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22f9f5f-a559-45e1-9920-fceadde8ee76-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:12.960812 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.960789    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e22f9f5f-a559-45e1-9920-fceadde8ee76" (UID: "e22f9f5f-a559-45e1-9920-fceadde8ee76"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:05:12.960954 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.960904    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e22f9f5f-a559-45e1-9920-fceadde8ee76-kube-api-access-n5kcl" (OuterVolumeSpecName: "kube-api-access-n5kcl") pod "e22f9f5f-a559-45e1-9920-fceadde8ee76" (UID: "e22f9f5f-a559-45e1-9920-fceadde8ee76"). InnerVolumeSpecName "kube-api-access-n5kcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:05:12.961268 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.961251    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5be5e0ea-f413-493f-a2b2-4ee588a37943-kube-api-access-cmnzn" (OuterVolumeSpecName: "kube-api-access-cmnzn") pod "5be5e0ea-f413-493f-a2b2-4ee588a37943" (UID: "5be5e0ea-f413-493f-a2b2-4ee588a37943"). InnerVolumeSpecName "kube-api-access-cmnzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:05:12.961307 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:12.961264    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7643acb1-46e6-4322-a31c-e46b9ce4a06d-kube-api-access-tbzhl" (OuterVolumeSpecName: "kube-api-access-tbzhl") pod "7643acb1-46e6-4322-a31c-e46b9ce4a06d" (UID: "7643acb1-46e6-4322-a31c-e46b9ce4a06d"). InnerVolumeSpecName "kube-api-access-tbzhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:05:13.060056 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.059979    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbzhl\" (UniqueName: \"kubernetes.io/projected/7643acb1-46e6-4322-a31c-e46b9ce4a06d-kube-api-access-tbzhl\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:13.060056 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.060007    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5kcl\" (UniqueName: \"kubernetes.io/projected/e22f9f5f-a559-45e1-9920-fceadde8ee76-kube-api-access-n5kcl\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:13.060056 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.060017    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmnzn\" (UniqueName: \"kubernetes.io/projected/5be5e0ea-f413-493f-a2b2-4ee588a37943-kube-api-access-cmnzn\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:13.060056 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.060027    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e22f9f5f-a559-45e1-9920-fceadde8ee76-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:13.794489 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.794417    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"
Jul 26 16:05:13.794489 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.794459    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb" event={"ID":"5be5e0ea-f413-493f-a2b2-4ee588a37943","Type":"ContainerDied","Data":"e61f82cf51f2013821a17e940930aa89829c6f0f54462deb7a5a750c2cb417f2"}
Jul 26 16:05:13.794965 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.794513    2560 scope.go:117] "RemoveContainer" containerID="d8fa9f76ba52741c1229260eaf1c32550bc1d5fb111d7f34a115c18e61868044"
Jul 26 16:05:13.795885 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.795837    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"
Jul 26 16:05:13.795885 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.795864    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84" event={"ID":"7643acb1-46e6-4322-a31c-e46b9ce4a06d","Type":"ContainerDied","Data":"a8ca307c19af04d7783837ce2ee3192edfaad66ac7b56e665e8188c517f6a0f9"}
Jul 26 16:05:13.797461 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.797439    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg" event={"ID":"e22f9f5f-a559-45e1-9920-fceadde8ee76","Type":"ContainerDied","Data":"19c56af916da0d2f43fdf38a288b8b5c70be838498c20556f89a5a51b42bd93d"}
Jul 26 16:05:13.797575 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.797467    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"
Jul 26 16:05:13.803583 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.803565    2560 scope.go:117] "RemoveContainer" containerID="33a5f52476a0ec29ccd72ae07d7667b60d31e7ca7df7a74fcc68086b58deb135"
Jul 26 16:05:13.810606 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.810590    2560 scope.go:117] "RemoveContainer" containerID="fb15718a26546674b62bdd86479a310f4eb41c7617b84e5b13c44e74b90192e3"
Jul 26 16:05:13.815412 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.815394    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"]
Jul 26 16:05:13.819113 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.819093    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-28d9404a-d8c58b79c-f5vpb"]
Jul 26 16:05:13.828126 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.828106    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"]
Jul 26 16:05:13.831860 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.831841    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-28d9404a-744b9c6c6d-sww84"]
Jul 26 16:05:13.840971 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.840952    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"]
Jul 26 16:05:13.847009 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:13.846991    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-28d9404a-6dd77dbcf8-8lxjg"]
Jul 26 16:05:15.139133 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:15.139102    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5be5e0ea-f413-493f-a2b2-4ee588a37943" path="/var/lib/kubelet/pods/5be5e0ea-f413-493f-a2b2-4ee588a37943/volumes"
Jul 26 16:05:15.139497 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:15.139448    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7643acb1-46e6-4322-a31c-e46b9ce4a06d" path="/var/lib/kubelet/pods/7643acb1-46e6-4322-a31c-e46b9ce4a06d/volumes"
Jul 26 16:05:15.139789 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:15.139775    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e22f9f5f-a559-45e1-9920-fceadde8ee76" path="/var/lib/kubelet/pods/e22f9f5f-a559-45e1-9920-fceadde8ee76/volumes"
Jul 26 16:05:17.089743 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:17.089696    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"]
Jul 26 16:05:17.092175 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:17.092151    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-28d9404a-f42bc"]
Jul 26 16:05:17.139957 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:17.139915    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ece6f91b-b4b9-42f1-98a9-9d448b1ebd27" path="/var/lib/kubelet/pods/ece6f91b-b4b9-42f1-98a9-9d448b1ebd27/volumes"
Jul 26 16:05:20.345001 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.344969    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"]
Jul 26 16:05:20.345396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345291    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be5e0ea-f413-493f-a2b2-4ee588a37943" containerName="payload-processing"
Jul 26 16:05:20.345396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345302    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be5e0ea-f413-493f-a2b2-4ee588a37943" containerName="payload-processing"
Jul 26 16:05:20.345396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345312    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e22f9f5f-a559-45e1-9920-fceadde8ee76" containerName="maas-api"
Jul 26 16:05:20.345396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345318    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e22f9f5f-a559-45e1-9920-fceadde8ee76" containerName="maas-api"
Jul 26 16:05:20.345396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345334    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7643acb1-46e6-4322-a31c-e46b9ce4a06d" containerName="payload-pre-processing"
Jul 26 16:05:20.345396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345340    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7643acb1-46e6-4322-a31c-e46b9ce4a06d" containerName="payload-pre-processing"
Jul 26 16:05:20.345396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345347    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece6f91b-b4b9-42f1-98a9-9d448b1ebd27" containerName="revoke-keys"
Jul 26 16:05:20.345396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345352    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece6f91b-b4b9-42f1-98a9-9d448b1ebd27" containerName="revoke-keys"
Jul 26 16:05:20.345396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345402    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be5e0ea-f413-493f-a2b2-4ee588a37943" containerName="payload-processing"
Jul 26 16:05:20.345724 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345410    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece6f91b-b4b9-42f1-98a9-9d448b1ebd27" containerName="revoke-keys"
Jul 26 16:05:20.345724 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345417    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7643acb1-46e6-4322-a31c-e46b9ce4a06d" containerName="payload-pre-processing"
Jul 26 16:05:20.345724 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.345423    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e22f9f5f-a559-45e1-9920-fceadde8ee76" containerName="maas-api"
Jul 26 16:05:20.349799 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.349778    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:20.352223 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.352195    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9823a2a6-serving-cert\""
Jul 26 16:05:20.367788 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.367755    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"]
Jul 26 16:05:20.422342 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.422310    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/179119b8-62d3-46ad-8056-e0be7d623874-openshift-service-ca\") pod \"maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:20.422534 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.422357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l88sh\" (UniqueName: \"kubernetes.io/projected/179119b8-62d3-46ad-8056-e0be7d623874-kube-api-access-l88sh\") pod \"maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:20.422534 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.422428    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179119b8-62d3-46ad-8056-e0be7d623874-maas-api-tls\") pod \"maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:20.523395 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.523363    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179119b8-62d3-46ad-8056-e0be7d623874-maas-api-tls\") pod \"maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:20.523541 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.523435    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/179119b8-62d3-46ad-8056-e0be7d623874-openshift-service-ca\") pod \"maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:20.523541 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.523464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l88sh\" (UniqueName: \"kubernetes.io/projected/179119b8-62d3-46ad-8056-e0be7d623874-kube-api-access-l88sh\") pod \"maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:20.523541 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:05:20.523507    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-serving-cert: secret "maas-api-e2e-ait-9823a2a6-serving-cert" not found
Jul 26 16:05:20.523695 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:05:20.523584    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179119b8-62d3-46ad-8056-e0be7d623874-maas-api-tls podName:179119b8-62d3-46ad-8056-e0be7d623874 nodeName:}" failed. No retries permitted until 2026-07-26 16:05:21.023566851 +0000 UTC m=+1636.522956667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/179119b8-62d3-46ad-8056-e0be7d623874-maas-api-tls") pod "maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9" (UID: "179119b8-62d3-46ad-8056-e0be7d623874") : secret "maas-api-e2e-ait-9823a2a6-serving-cert" not found
Jul 26 16:05:20.524154 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.524136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/179119b8-62d3-46ad-8056-e0be7d623874-openshift-service-ca\") pod \"maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:20.536323 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.536304    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l88sh\" (UniqueName: \"kubernetes.io/projected/179119b8-62d3-46ad-8056-e0be7d623874-kube-api-access-l88sh\") pod \"maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:20.994088 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.994051    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"]
Jul 26 16:05:20.997691 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.997674    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:20.999827 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.999796    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9823a2a6\""
Jul 26 16:05:20.999962 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:20.999939    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-9823a2a6-dockercfg-4hq5r\""
Jul 26 16:05:21.014186 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.014156    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"]
Jul 26 16:05:21.023324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.023296    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"]
Jul 26 16:05:21.026780 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.026758    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:21.027404 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.027379    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179119b8-62d3-46ad-8056-e0be7d623874-maas-api-tls\") pod \"maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:21.027529 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.027473    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z8k8\" (UniqueName: \"kubernetes.io/projected/245e25da-410d-4ef3-a11e-ed8c3eb88feb-kube-api-access-2z8k8\") pod \"payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh\" (UID: \"245e25da-410d-4ef3-a11e-ed8c3eb88feb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:21.027611 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.027529    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/245e25da-410d-4ef3-a11e-ed8c3eb88feb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh\" (UID: \"245e25da-410d-4ef3-a11e-ed8c3eb88feb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:21.029880 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.029851    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179119b8-62d3-46ad-8056-e0be7d623874-maas-api-tls\") pod \"maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:21.037021 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.036686    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"]
Jul 26 16:05:21.128277 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.128233    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8k8\" (UniqueName: \"kubernetes.io/projected/245e25da-410d-4ef3-a11e-ed8c3eb88feb-kube-api-access-2z8k8\") pod \"payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh\" (UID: \"245e25da-410d-4ef3-a11e-ed8c3eb88feb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:21.128475 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.128310    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/245e25da-410d-4ef3-a11e-ed8c3eb88feb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh\" (UID: \"245e25da-410d-4ef3-a11e-ed8c3eb88feb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:21.128475 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.128346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-plugins-config-volume\") pod \"payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf\" (UID: \"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f\") " pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:21.128475 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.128387    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r2pg\" (UniqueName: \"kubernetes.io/projected/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-kube-api-access-9r2pg\") pod \"payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf\" (UID: \"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f\") " pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:21.129077 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.129051    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/245e25da-410d-4ef3-a11e-ed8c3eb88feb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh\" (UID: \"245e25da-410d-4ef3-a11e-ed8c3eb88feb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:21.137274 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.137251    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z8k8\" (UniqueName: \"kubernetes.io/projected/245e25da-410d-4ef3-a11e-ed8c3eb88feb-kube-api-access-2z8k8\") pod \"payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh\" (UID: \"245e25da-410d-4ef3-a11e-ed8c3eb88feb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:21.229098 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.229057    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-plugins-config-volume\") pod \"payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf\" (UID: \"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f\") " pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:21.229098 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.229105    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r2pg\" (UniqueName: \"kubernetes.io/projected/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-kube-api-access-9r2pg\") pod \"payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf\" (UID: \"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f\") " pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:21.229978 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.229930    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-plugins-config-volume\") pod \"payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf\" (UID: \"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f\") " pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:21.236803 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.236776    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r2pg\" (UniqueName: \"kubernetes.io/projected/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-kube-api-access-9r2pg\") pod \"payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf\" (UID: \"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f\") " pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:21.259677 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.259612    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:21.308859 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.308727    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:21.345794 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.345761    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:21.415384 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.414295    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"]
Jul 26 16:05:21.419517 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:05:21.419191    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179119b8_62d3_46ad_8056_e0be7d623874.slice/crio-cf701262ca6904c12be52e77d2ea2b78c5a050200bd9dc6cd7576bc8d9ccec0c WatchSource:0}: Error finding container cf701262ca6904c12be52e77d2ea2b78c5a050200bd9dc6cd7576bc8d9ccec0c: Status 404 returned error can't find the container with id cf701262ca6904c12be52e77d2ea2b78c5a050200bd9dc6cd7576bc8d9ccec0c
Jul 26 16:05:21.499444 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.499383    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"]
Jul 26 16:05:21.502936 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:05:21.502906    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245e25da_410d_4ef3_a11e_ed8c3eb88feb.slice/crio-2ceae93154b10bda948a9e99d64a6e88855c94f5569d70d7272195249402a907 WatchSource:0}: Error finding container 2ceae93154b10bda948a9e99d64a6e88855c94f5569d70d7272195249402a907: Status 404 returned error can't find the container with id 2ceae93154b10bda948a9e99d64a6e88855c94f5569d70d7272195249402a907
Jul 26 16:05:21.745409 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.745370    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"]
Jul 26 16:05:21.747122 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:05:21.747094    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62faef55_a5f7_4bb6_b1c3_fc32fa3b9e0f.slice/crio-0027b2a1ec0d4ef02cf786731907df047724b56ec2280ab8ca58c92d91a82774 WatchSource:0}: Error finding container 0027b2a1ec0d4ef02cf786731907df047724b56ec2280ab8ca58c92d91a82774: Status 404 returned error can't find the container with id 0027b2a1ec0d4ef02cf786731907df047724b56ec2280ab8ca58c92d91a82774
Jul 26 16:05:21.838764 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.838729    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh" event={"ID":"245e25da-410d-4ef3-a11e-ed8c3eb88feb","Type":"ContainerStarted","Data":"0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805"}
Jul 26 16:05:21.838869 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.838819    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh" event={"ID":"245e25da-410d-4ef3-a11e-ed8c3eb88feb","Type":"ContainerStarted","Data":"2ceae93154b10bda948a9e99d64a6e88855c94f5569d70d7272195249402a907"}
Jul 26 16:05:21.838963 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.838934    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:21.840009 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.839987    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9" event={"ID":"179119b8-62d3-46ad-8056-e0be7d623874","Type":"ContainerStarted","Data":"cf701262ca6904c12be52e77d2ea2b78c5a050200bd9dc6cd7576bc8d9ccec0c"}
Jul 26 16:05:21.841389 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.841367    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf" event={"ID":"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f","Type":"ContainerStarted","Data":"6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4"}
Jul 26 16:05:21.841488 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.841424    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf" event={"ID":"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f","Type":"ContainerStarted","Data":"0027b2a1ec0d4ef02cf786731907df047724b56ec2280ab8ca58c92d91a82774"}
Jul 26 16:05:21.841488 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.841465    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:21.854081 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.854040    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh" podStartSLOduration=1.854024863 podStartE2EDuration="1.854024863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:05:21.852508016 +0000 UTC m=+1637.351897854" watchObservedRunningTime="2026-07-26 16:05:21.854024863 +0000 UTC m=+1637.353414700"
Jul 26 16:05:21.867522 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:21.867474    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf" podStartSLOduration=1.867456168 podStartE2EDuration="1.867456168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:05:21.867031093 +0000 UTC m=+1637.366420930" watchObservedRunningTime="2026-07-26 16:05:21.867456168 +0000 UTC m=+1637.366846012"
Jul 26 16:05:22.847689 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:22.847646    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9" event={"ID":"179119b8-62d3-46ad-8056-e0be7d623874","Type":"ContainerStarted","Data":"d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516"}
Jul 26 16:05:22.848137 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:22.847990    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:22.864238 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:22.864189    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9" podStartSLOduration=2.469895724 podStartE2EDuration="2.864173011s" totalImagesPullingTime="394.277287ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:05:22.862705344 +0000 UTC m=+1638.362095182" watchObservedRunningTime="2026-07-26 16:05:22.864173011 +0000 UTC m=+1638.363562886"
Jul 26 16:05:28.858108 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:28.858076    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:32.850278 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:32.850246    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:32.850682 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:32.850454    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:45.906499 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:45.906467    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"]
Jul 26 16:05:45.909858 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:45.909838    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"
Jul 26 16:05:45.911805 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:45.911785    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:05:45.919821 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:45.919797    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"]
Jul 26 16:05:46.037227 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:46.037177    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d4c354-f2a1-4f01-89f5-012204baf5ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4\" (UID: \"e8d4c354-f2a1-4f01-89f5-012204baf5ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"
Jul 26 16:05:46.138668 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:46.138632    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d4c354-f2a1-4f01-89f5-012204baf5ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4\" (UID: \"e8d4c354-f2a1-4f01-89f5-012204baf5ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"
Jul 26 16:05:46.139263 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:46.139244    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d4c354-f2a1-4f01-89f5-012204baf5ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4\" (UID: \"e8d4c354-f2a1-4f01-89f5-012204baf5ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"
Jul 26 16:05:46.221410 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:46.221318    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"
Jul 26 16:05:46.351440 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:46.350746    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"]
Jul 26 16:05:46.357469 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:05:46.357437    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d4c354_f2a1_4f01_89f5_012204baf5ee.slice/crio-b84326434a2415113bc79a83f9903678c3a4d3dc59d3bde54f12a62bdeb9f311 WatchSource:0}: Error finding container b84326434a2415113bc79a83f9903678c3a4d3dc59d3bde54f12a62bdeb9f311: Status 404 returned error can't find the container with id b84326434a2415113bc79a83f9903678c3a4d3dc59d3bde54f12a62bdeb9f311
Jul 26 16:05:46.957040 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:46.957007    2560 generic.go:356] "Generic (PLEG): container finished" podID="e8d4c354-f2a1-4f01-89f5-012204baf5ee" containerID="8396549ae4d8f9f6405f93ef714c26877e55b9e5b946d3c516794528b79936f7" exitCode=0
Jul 26 16:05:46.957371 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:46.957086    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4" event={"ID":"e8d4c354-f2a1-4f01-89f5-012204baf5ee","Type":"ContainerDied","Data":"8396549ae4d8f9f6405f93ef714c26877e55b9e5b946d3c516794528b79936f7"}
Jul 26 16:05:46.957371 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:46.957137    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4" event={"ID":"e8d4c354-f2a1-4f01-89f5-012204baf5ee","Type":"ContainerStarted","Data":"b84326434a2415113bc79a83f9903678c3a4d3dc59d3bde54f12a62bdeb9f311"}
Jul 26 16:05:48.087808 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:48.087784    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"
Jul 26 16:05:48.157342 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:48.157316    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d4c354-f2a1-4f01-89f5-012204baf5ee-maas-api-service-ca\") pod \"e8d4c354-f2a1-4f01-89f5-012204baf5ee\" (UID: \"e8d4c354-f2a1-4f01-89f5-012204baf5ee\") "
Jul 26 16:05:48.157646 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:48.157625    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d4c354-f2a1-4f01-89f5-012204baf5ee-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8d4c354-f2a1-4f01-89f5-012204baf5ee" (UID: "e8d4c354-f2a1-4f01-89f5-012204baf5ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:05:48.258096 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:48.258030    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d4c354-f2a1-4f01-89f5-012204baf5ee-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:48.967016 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:48.966991    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"
Jul 26 16:05:48.967178 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:48.966989    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4" event={"ID":"e8d4c354-f2a1-4f01-89f5-012204baf5ee","Type":"ContainerDied","Data":"b84326434a2415113bc79a83f9903678c3a4d3dc59d3bde54f12a62bdeb9f311"}
Jul 26 16:05:48.967178 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:48.967117    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b84326434a2415113bc79a83f9903678c3a4d3dc59d3bde54f12a62bdeb9f311"
Jul 26 16:05:56.294108 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.294071    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"]
Jul 26 16:05:56.294516 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.294305    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9" podUID="179119b8-62d3-46ad-8056-e0be7d623874" containerName="maas-api" containerID="cri-o://d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516" gracePeriod=30
Jul 26 16:05:56.353407 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.353372    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"]
Jul 26 16:05:56.353630 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.353603    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf" podUID="62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f" containerName="payload-processing" containerID="cri-o://6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4" gracePeriod=30
Jul 26 16:05:56.360345 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.360321    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"]
Jul 26 16:05:56.360510 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.360490    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh" podUID="245e25da-410d-4ef3-a11e-ed8c3eb88feb" containerName="payload-pre-processing" containerID="cri-o://0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805" gracePeriod=30
Jul 26 16:05:56.647056 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.647028    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:56.649064 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.649043    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:56.651758 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.651741    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:56.721794 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.721768    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l88sh\" (UniqueName: \"kubernetes.io/projected/179119b8-62d3-46ad-8056-e0be7d623874-kube-api-access-l88sh\") pod \"179119b8-62d3-46ad-8056-e0be7d623874\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") "
Jul 26 16:05:56.721962 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.721816    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/245e25da-410d-4ef3-a11e-ed8c3eb88feb-plugins-config-volume\") pod \"245e25da-410d-4ef3-a11e-ed8c3eb88feb\" (UID: \"245e25da-410d-4ef3-a11e-ed8c3eb88feb\") "
Jul 26 16:05:56.721962 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.721856    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/179119b8-62d3-46ad-8056-e0be7d623874-openshift-service-ca\") pod \"179119b8-62d3-46ad-8056-e0be7d623874\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") "
Jul 26 16:05:56.721962 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.721883    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r2pg\" (UniqueName: \"kubernetes.io/projected/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-kube-api-access-9r2pg\") pod \"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f\" (UID: \"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f\") "
Jul 26 16:05:56.721962 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.721943    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179119b8-62d3-46ad-8056-e0be7d623874-maas-api-tls\") pod \"179119b8-62d3-46ad-8056-e0be7d623874\" (UID: \"179119b8-62d3-46ad-8056-e0be7d623874\") "
Jul 26 16:05:56.722178 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.721971    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z8k8\" (UniqueName: \"kubernetes.io/projected/245e25da-410d-4ef3-a11e-ed8c3eb88feb-kube-api-access-2z8k8\") pod \"245e25da-410d-4ef3-a11e-ed8c3eb88feb\" (UID: \"245e25da-410d-4ef3-a11e-ed8c3eb88feb\") "
Jul 26 16:05:56.722178 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.722028    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-plugins-config-volume\") pod \"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f\" (UID: \"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f\") "
Jul 26 16:05:56.722320 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.722285    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/245e25da-410d-4ef3-a11e-ed8c3eb88feb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "245e25da-410d-4ef3-a11e-ed8c3eb88feb" (UID: "245e25da-410d-4ef3-a11e-ed8c3eb88feb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:05:56.722872 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.722841    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/179119b8-62d3-46ad-8056-e0be7d623874-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "179119b8-62d3-46ad-8056-e0be7d623874" (UID: "179119b8-62d3-46ad-8056-e0be7d623874"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:05:56.723009 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.722866    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f" (UID: "62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:05:56.724154 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.724129    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/179119b8-62d3-46ad-8056-e0be7d623874-kube-api-access-l88sh" (OuterVolumeSpecName: "kube-api-access-l88sh") pod "179119b8-62d3-46ad-8056-e0be7d623874" (UID: "179119b8-62d3-46ad-8056-e0be7d623874"). InnerVolumeSpecName "kube-api-access-l88sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:05:56.724246 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.724190    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-kube-api-access-9r2pg" (OuterVolumeSpecName: "kube-api-access-9r2pg") pod "62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f" (UID: "62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f"). InnerVolumeSpecName "kube-api-access-9r2pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:05:56.724332 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.724312    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/179119b8-62d3-46ad-8056-e0be7d623874-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "179119b8-62d3-46ad-8056-e0be7d623874" (UID: "179119b8-62d3-46ad-8056-e0be7d623874"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:05:56.724804 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.724779    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/245e25da-410d-4ef3-a11e-ed8c3eb88feb-kube-api-access-2z8k8" (OuterVolumeSpecName: "kube-api-access-2z8k8") pod "245e25da-410d-4ef3-a11e-ed8c3eb88feb" (UID: "245e25da-410d-4ef3-a11e-ed8c3eb88feb"). InnerVolumeSpecName "kube-api-access-2z8k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:05:56.823397 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.823374    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:56.823397 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.823396    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l88sh\" (UniqueName: \"kubernetes.io/projected/179119b8-62d3-46ad-8056-e0be7d623874-kube-api-access-l88sh\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:56.823521 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.823405    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/245e25da-410d-4ef3-a11e-ed8c3eb88feb-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:56.823521 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.823414    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/179119b8-62d3-46ad-8056-e0be7d623874-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:56.823521 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.823422    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r2pg\" (UniqueName: \"kubernetes.io/projected/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f-kube-api-access-9r2pg\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:56.823521 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.823431    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179119b8-62d3-46ad-8056-e0be7d623874-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:56.823521 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:56.823439    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z8k8\" (UniqueName: \"kubernetes.io/projected/245e25da-410d-4ef3-a11e-ed8c3eb88feb-kube-api-access-2z8k8\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:05:57.002653 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.002631    2560 generic.go:356] "Generic (PLEG): container finished" podID="245e25da-410d-4ef3-a11e-ed8c3eb88feb" containerID="0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805" exitCode=0
Jul 26 16:05:57.002763 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.002676    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"
Jul 26 16:05:57.002763 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.002697    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh" event={"ID":"245e25da-410d-4ef3-a11e-ed8c3eb88feb","Type":"ContainerDied","Data":"0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805"}
Jul 26 16:05:57.002763 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.002739    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh" event={"ID":"245e25da-410d-4ef3-a11e-ed8c3eb88feb","Type":"ContainerDied","Data":"2ceae93154b10bda948a9e99d64a6e88855c94f5569d70d7272195249402a907"}
Jul 26 16:05:57.002917 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.002768    2560 scope.go:117] "RemoveContainer" containerID="0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805"
Jul 26 16:05:57.004038 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.004012    2560 generic.go:356] "Generic (PLEG): container finished" podID="179119b8-62d3-46ad-8056-e0be7d623874" containerID="d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516" exitCode=0
Jul 26 16:05:57.004129 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.004044    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9" event={"ID":"179119b8-62d3-46ad-8056-e0be7d623874","Type":"ContainerDied","Data":"d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516"}
Jul 26 16:05:57.004129 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.004103    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9" event={"ID":"179119b8-62d3-46ad-8056-e0be7d623874","Type":"ContainerDied","Data":"cf701262ca6904c12be52e77d2ea2b78c5a050200bd9dc6cd7576bc8d9ccec0c"}
Jul 26 16:05:57.004129 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.004057    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"
Jul 26 16:05:57.005414 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.005389    2560 generic.go:356] "Generic (PLEG): container finished" podID="62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f" containerID="6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4" exitCode=0
Jul 26 16:05:57.005520 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.005424    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"
Jul 26 16:05:57.005520 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.005450    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf" event={"ID":"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f","Type":"ContainerDied","Data":"6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4"}
Jul 26 16:05:57.005520 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.005487    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf" event={"ID":"62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f","Type":"ContainerDied","Data":"0027b2a1ec0d4ef02cf786731907df047724b56ec2280ab8ca58c92d91a82774"}
Jul 26 16:05:57.013220 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.013202    2560 scope.go:117] "RemoveContainer" containerID="0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805"
Jul 26 16:05:57.013496 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:05:57.013471    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805\": container with ID starting with 0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805 not found: ID does not exist" containerID="0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805"
Jul 26 16:05:57.013592 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.013503    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805"} err="failed to get container status \"0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805\": rpc error: code = NotFound desc = could not find container \"0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805\": container with ID starting with 0c7eb51592ba4a4ec1d8b05da8a0ca255a7d5e409f198b45223dbaa1f6ee3805 not found: ID does not exist"
Jul 26 16:05:57.013592 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.013526    2560 scope.go:117] "RemoveContainer" containerID="d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516"
Jul 26 16:05:57.022165 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.022152    2560 scope.go:117] "RemoveContainer" containerID="d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516"
Jul 26 16:05:57.022393 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:05:57.022377    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516\": container with ID starting with d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516 not found: ID does not exist" containerID="d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516"
Jul 26 16:05:57.022434 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.022400    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516"} err="failed to get container status \"d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516\": rpc error: code = NotFound desc = could not find container \"d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516\": container with ID starting with d4e7cfb93eed264affde4bcf164af2b08d843c15dea2eea161dfa0e81eac3516 not found: ID does not exist"
Jul 26 16:05:57.022434 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.022415    2560 scope.go:117] "RemoveContainer" containerID="6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4"
Jul 26 16:05:57.029424 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.029407    2560 scope.go:117] "RemoveContainer" containerID="6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4"
Jul 26 16:05:57.029653 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:05:57.029637    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4\": container with ID starting with 6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4 not found: ID does not exist" containerID="6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4"
Jul 26 16:05:57.029693 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.029659    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4"} err="failed to get container status \"6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4\": rpc error: code = NotFound desc = could not find container \"6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4\": container with ID starting with 6d62c0cbd6876a333e9e8ac6d1e99591e7dc66c3bd876a29e1fa68d2ce1faeb4 not found: ID does not exist"
Jul 26 16:05:57.046956 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.046933    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"]
Jul 26 16:05:57.060758 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.060736    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9823a2a6-57cd587796-tbllh"]
Jul 26 16:05:57.090370 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.090348    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"]
Jul 26 16:05:57.093868 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.093850    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9823a2a6-5bd58f7bcb-kvjnf"]
Jul 26 16:05:57.118547 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.118527    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"]
Jul 26 16:05:57.126984 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.126964    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9823a2a6-6c84b76bd7-f46t9"]
Jul 26 16:05:57.143780 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.143755    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="179119b8-62d3-46ad-8056-e0be7d623874" path="/var/lib/kubelet/pods/179119b8-62d3-46ad-8056-e0be7d623874/volumes"
Jul 26 16:05:57.144101 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.144088    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="245e25da-410d-4ef3-a11e-ed8c3eb88feb" path="/var/lib/kubelet/pods/245e25da-410d-4ef3-a11e-ed8c3eb88feb/volumes"
Jul 26 16:05:57.144389 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:05:57.144378    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f" path="/var/lib/kubelet/pods/62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f/volumes"
Jul 26 16:06:00.973321 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:00.973288    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"]
Jul 26 16:06:00.976379 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:00.976355    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9823a2a6-ktlr4"]
Jul 26 16:06:01.138673 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:01.138646    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8d4c354-f2a1-4f01-89f5-012204baf5ee" path="/var/lib/kubelet/pods/e8d4c354-f2a1-4f01-89f5-012204baf5ee/volumes"
Jul 26 16:06:05.768363 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.768325    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"]
Jul 26 16:06:05.768844 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.768829    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d4c354-f2a1-4f01-89f5-012204baf5ee" containerName="revoke-keys"
Jul 26 16:06:05.768913 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.768848    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d4c354-f2a1-4f01-89f5-012204baf5ee" containerName="revoke-keys"
Jul 26 16:06:05.768913 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.768871    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="179119b8-62d3-46ad-8056-e0be7d623874" containerName="maas-api"
Jul 26 16:06:05.768913 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.768880    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="179119b8-62d3-46ad-8056-e0be7d623874" containerName="maas-api"
Jul 26 16:06:05.769016 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.768925    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f" containerName="payload-processing"
Jul 26 16:06:05.769016 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.768935    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f" containerName="payload-processing"
Jul 26 16:06:05.769016 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.768956    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="245e25da-410d-4ef3-a11e-ed8c3eb88feb" containerName="payload-pre-processing"
Jul 26 16:06:05.769016 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.768965    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="245e25da-410d-4ef3-a11e-ed8c3eb88feb" containerName="payload-pre-processing"
Jul 26 16:06:05.769147 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.769066    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="62faef55-a5f7-4bb6-b1c3-fc32fa3b9e0f" containerName="payload-processing"
Jul 26 16:06:05.769147 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.769078    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="245e25da-410d-4ef3-a11e-ed8c3eb88feb" containerName="payload-pre-processing"
Jul 26 16:06:05.769147 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.769088    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d4c354-f2a1-4f01-89f5-012204baf5ee" containerName="revoke-keys"
Jul 26 16:06:05.769147 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.769097    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="179119b8-62d3-46ad-8056-e0be7d623874" containerName="maas-api"
Jul 26 16:06:05.774067 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.774047    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:05.775928 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.775900    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c17951b0-serving-cert\""
Jul 26 16:06:05.783436 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.783339    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"]
Jul 26 16:06:05.892449 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.892420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024f6057-1134-4945-bc33-1798a39cf6b4-openshift-service-ca\") pod \"maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:05.892586 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.892457    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-844b9\" (UniqueName: \"kubernetes.io/projected/024f6057-1134-4945-bc33-1798a39cf6b4-kube-api-access-844b9\") pod \"maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:05.892586 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.892569    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024f6057-1134-4945-bc33-1798a39cf6b4-maas-api-tls\") pod \"maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:05.993187 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.993160    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024f6057-1134-4945-bc33-1798a39cf6b4-maas-api-tls\") pod \"maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:05.993300 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.993212    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024f6057-1134-4945-bc33-1798a39cf6b4-openshift-service-ca\") pod \"maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:05.993300 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.993258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-844b9\" (UniqueName: \"kubernetes.io/projected/024f6057-1134-4945-bc33-1798a39cf6b4-kube-api-access-844b9\") pod \"maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:05.993730 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.993712    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024f6057-1134-4945-bc33-1798a39cf6b4-openshift-service-ca\") pod \"maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:05.995591 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:05.995573    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024f6057-1134-4945-bc33-1798a39cf6b4-maas-api-tls\") pod \"maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:06.007183 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.007158    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-844b9\" (UniqueName: \"kubernetes.io/projected/024f6057-1134-4945-bc33-1798a39cf6b4-kube-api-access-844b9\") pod \"maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:06.087659 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.087595    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:06.215335 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.215312    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"]
Jul 26 16:06:06.217814 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:06:06.217786    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024f6057_1134_4945_bc33_1798a39cf6b4.slice/crio-d830357ff279aa30f10f4a24e43731042f2fb267b818d4489457492f548ddb34 WatchSource:0}: Error finding container d830357ff279aa30f10f4a24e43731042f2fb267b818d4489457492f548ddb34: Status 404 returned error can't find the container with id d830357ff279aa30f10f4a24e43731042f2fb267b818d4489457492f548ddb34
Jul 26 16:06:06.472244 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.472212    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"]
Jul 26 16:06:06.476777 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.476761    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:06.479009 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.478991    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-c17951b0\""
Jul 26 16:06:06.479469 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.479455    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-c17951b0-dockercfg-nvqp9\""
Jul 26 16:06:06.485571 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.485548    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"]
Jul 26 16:06:06.508362 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.508341    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"]
Jul 26 16:06:06.511806 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.511783    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:06.529104 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.529085    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"]
Jul 26 16:06:06.597622 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.597592    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlzsl\" (UniqueName: \"kubernetes.io/projected/c3307aa3-c0ec-4a7e-85a0-12173a05332c-kube-api-access-qlzsl\") pod \"payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m\" (UID: \"c3307aa3-c0ec-4a7e-85a0-12173a05332c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:06.597771 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.597695    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52f11902-4b34-4147-8d76-26d09af45dcb-plugins-config-volume\") pod \"payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j\" (UID: \"52f11902-4b34-4147-8d76-26d09af45dcb\") " pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:06.597842 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.597765    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3307aa3-c0ec-4a7e-85a0-12173a05332c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m\" (UID: \"c3307aa3-c0ec-4a7e-85a0-12173a05332c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:06.597907 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.597851    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slglj\" (UniqueName: \"kubernetes.io/projected/52f11902-4b34-4147-8d76-26d09af45dcb-kube-api-access-slglj\") pod \"payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j\" (UID: \"52f11902-4b34-4147-8d76-26d09af45dcb\") " pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:06.699020 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.698987    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slglj\" (UniqueName: \"kubernetes.io/projected/52f11902-4b34-4147-8d76-26d09af45dcb-kube-api-access-slglj\") pod \"payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j\" (UID: \"52f11902-4b34-4147-8d76-26d09af45dcb\") " pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:06.699020 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.699026    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzsl\" (UniqueName: \"kubernetes.io/projected/c3307aa3-c0ec-4a7e-85a0-12173a05332c-kube-api-access-qlzsl\") pod \"payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m\" (UID: \"c3307aa3-c0ec-4a7e-85a0-12173a05332c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:06.699223 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.699069    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52f11902-4b34-4147-8d76-26d09af45dcb-plugins-config-volume\") pod \"payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j\" (UID: \"52f11902-4b34-4147-8d76-26d09af45dcb\") " pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:06.699223 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.699102    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3307aa3-c0ec-4a7e-85a0-12173a05332c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m\" (UID: \"c3307aa3-c0ec-4a7e-85a0-12173a05332c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:06.699833 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.699807    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52f11902-4b34-4147-8d76-26d09af45dcb-plugins-config-volume\") pod \"payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j\" (UID: \"52f11902-4b34-4147-8d76-26d09af45dcb\") " pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:06.699963 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.699813    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3307aa3-c0ec-4a7e-85a0-12173a05332c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m\" (UID: \"c3307aa3-c0ec-4a7e-85a0-12173a05332c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:06.709360 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.709337    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlzsl\" (UniqueName: \"kubernetes.io/projected/c3307aa3-c0ec-4a7e-85a0-12173a05332c-kube-api-access-qlzsl\") pod \"payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m\" (UID: \"c3307aa3-c0ec-4a7e-85a0-12173a05332c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:06.709583 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.709564    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slglj\" (UniqueName: \"kubernetes.io/projected/52f11902-4b34-4147-8d76-26d09af45dcb-kube-api-access-slglj\") pod \"payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j\" (UID: \"52f11902-4b34-4147-8d76-26d09af45dcb\") " pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:06.788621 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.788540    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:06.820909 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.820860    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:06.938643 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:06.938559    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"]
Jul 26 16:06:06.943461 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:06:06.943434    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3307aa3_c0ec_4a7e_85a0_12173a05332c.slice/crio-00e9de22f7ca339076b2503c4d6ed1ac28185d8294cb34827bbc378b4e661dd7 WatchSource:0}: Error finding container 00e9de22f7ca339076b2503c4d6ed1ac28185d8294cb34827bbc378b4e661dd7: Status 404 returned error can't find the container with id 00e9de22f7ca339076b2503c4d6ed1ac28185d8294cb34827bbc378b4e661dd7
Jul 26 16:06:07.054558 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:07.054492    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g" event={"ID":"024f6057-1134-4945-bc33-1798a39cf6b4","Type":"ContainerStarted","Data":"59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337"}
Jul 26 16:06:07.054670 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:07.054654    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:07.054723 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:07.054691    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g" event={"ID":"024f6057-1134-4945-bc33-1798a39cf6b4","Type":"ContainerStarted","Data":"d830357ff279aa30f10f4a24e43731042f2fb267b818d4489457492f548ddb34"}
Jul 26 16:06:07.055952 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:07.055923    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m" event={"ID":"c3307aa3-c0ec-4a7e-85a0-12173a05332c","Type":"ContainerStarted","Data":"7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a"}
Jul 26 16:06:07.056053 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:07.055967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m" event={"ID":"c3307aa3-c0ec-4a7e-85a0-12173a05332c","Type":"ContainerStarted","Data":"00e9de22f7ca339076b2503c4d6ed1ac28185d8294cb34827bbc378b4e661dd7"}
Jul 26 16:06:07.056053 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:07.055989    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:07.069269 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:07.069217    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g" podStartSLOduration=1.321805713 podStartE2EDuration="2.06920213s" totalImagesPullingTime="747.396417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:06:07.068475264 +0000 UTC m=+1682.567865104" watchObservedRunningTime="2026-07-26 16:06:07.06920213 +0000 UTC m=+1682.568591968"
Jul 26 16:06:07.082395 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:07.082356    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m" podStartSLOduration=1.08233935 podStartE2EDuration="1.08233935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:06:07.081847655 +0000 UTC m=+1682.581237493" watchObservedRunningTime="2026-07-26 16:06:07.08233935 +0000 UTC m=+1682.581729189"
Jul 26 16:06:07.174854 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:07.174818    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"]
Jul 26 16:06:07.177816 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:06:07.177780    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f11902_4b34_4147_8d76_26d09af45dcb.slice/crio-543be14d046948e3b9ee20ec493623235a9cf2e7a248ebe6ff9be8dee7806860 WatchSource:0}: Error finding container 543be14d046948e3b9ee20ec493623235a9cf2e7a248ebe6ff9be8dee7806860: Status 404 returned error can't find the container with id 543be14d046948e3b9ee20ec493623235a9cf2e7a248ebe6ff9be8dee7806860
Jul 26 16:06:08.062966 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:08.062922    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j" event={"ID":"52f11902-4b34-4147-8d76-26d09af45dcb","Type":"ContainerStarted","Data":"416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2"}
Jul 26 16:06:08.063481 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:08.062980    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j" event={"ID":"52f11902-4b34-4147-8d76-26d09af45dcb","Type":"ContainerStarted","Data":"543be14d046948e3b9ee20ec493623235a9cf2e7a248ebe6ff9be8dee7806860"}
Jul 26 16:06:08.063481 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:08.063153    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:08.078128 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:08.078077    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j" podStartSLOduration=2.078060752 podStartE2EDuration="2.078060752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:06:08.07730412 +0000 UTC m=+1683.576693959" watchObservedRunningTime="2026-07-26 16:06:08.078060752 +0000 UTC m=+1683.577450589"
Jul 26 16:06:13.068729 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:13.068697    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:18.066045 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:18.066006    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:19.071075 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:19.071041    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:30.339653 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:30.339571    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"]
Jul 26 16:06:30.343071 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:30.343050    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"
Jul 26 16:06:30.344911 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:30.344877    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:06:30.348836 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:30.348813    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"]
Jul 26 16:06:30.397534 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:30.397502    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c49cd25-d42e-4374-aa24-6fa341f2d7b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx\" (UID: \"0c49cd25-d42e-4374-aa24-6fa341f2d7b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"
Jul 26 16:06:30.498565 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:30.498530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c49cd25-d42e-4374-aa24-6fa341f2d7b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx\" (UID: \"0c49cd25-d42e-4374-aa24-6fa341f2d7b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"
Jul 26 16:06:30.499153 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:30.499135    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c49cd25-d42e-4374-aa24-6fa341f2d7b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx\" (UID: \"0c49cd25-d42e-4374-aa24-6fa341f2d7b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"
Jul 26 16:06:30.655838 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:30.655800    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"
Jul 26 16:06:30.787267 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:30.787240    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"]
Jul 26 16:06:30.789819 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:06:30.789793    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c49cd25_d42e_4374_aa24_6fa341f2d7b3.slice/crio-a3db888e03a5da54ad83190109fbd0be2bf21f3d52728b2857ca9133a3c6d87d WatchSource:0}: Error finding container a3db888e03a5da54ad83190109fbd0be2bf21f3d52728b2857ca9133a3c6d87d: Status 404 returned error can't find the container with id a3db888e03a5da54ad83190109fbd0be2bf21f3d52728b2857ca9133a3c6d87d
Jul 26 16:06:31.168265 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:31.168224    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx" event={"ID":"0c49cd25-d42e-4374-aa24-6fa341f2d7b3","Type":"ContainerStarted","Data":"ed160cca90b8648551a91716b3cf9e3ead0b6a6ddd8f74bafe7c90fd1c0a7dad"}
Jul 26 16:06:31.168443 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:31.168287    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx" event={"ID":"0c49cd25-d42e-4374-aa24-6fa341f2d7b3","Type":"ContainerStarted","Data":"a3db888e03a5da54ad83190109fbd0be2bf21f3d52728b2857ca9133a3c6d87d"}
Jul 26 16:06:31.183513 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:31.183463    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx" podStartSLOduration=1.183447795 podStartE2EDuration="1.183447795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:06:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:06:31.182690227 +0000 UTC m=+1706.682080066" watchObservedRunningTime="2026-07-26 16:06:31.183447795 +0000 UTC m=+1706.682837632"
Jul 26 16:06:32.174431 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:32.174397    2560 generic.go:356] "Generic (PLEG): container finished" podID="0c49cd25-d42e-4374-aa24-6fa341f2d7b3" containerID="ed160cca90b8648551a91716b3cf9e3ead0b6a6ddd8f74bafe7c90fd1c0a7dad" exitCode=0
Jul 26 16:06:32.174858 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:32.174489    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx" event={"ID":"0c49cd25-d42e-4374-aa24-6fa341f2d7b3","Type":"ContainerDied","Data":"ed160cca90b8648551a91716b3cf9e3ead0b6a6ddd8f74bafe7c90fd1c0a7dad"}
Jul 26 16:06:33.325907 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:33.325874    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"
Jul 26 16:06:33.425343 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:33.425309    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c49cd25-d42e-4374-aa24-6fa341f2d7b3-maas-api-service-ca\") pod \"0c49cd25-d42e-4374-aa24-6fa341f2d7b3\" (UID: \"0c49cd25-d42e-4374-aa24-6fa341f2d7b3\") "
Jul 26 16:06:33.425691 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:33.425665    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c49cd25-d42e-4374-aa24-6fa341f2d7b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c49cd25-d42e-4374-aa24-6fa341f2d7b3" (UID: "0c49cd25-d42e-4374-aa24-6fa341f2d7b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:06:33.526571 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:33.526482    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c49cd25-d42e-4374-aa24-6fa341f2d7b3-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:06:34.184608 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:34.184576    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"
Jul 26 16:06:34.184608 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:34.184585    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx" event={"ID":"0c49cd25-d42e-4374-aa24-6fa341f2d7b3","Type":"ContainerDied","Data":"a3db888e03a5da54ad83190109fbd0be2bf21f3d52728b2857ca9133a3c6d87d"}
Jul 26 16:06:34.184807 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:34.184638    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3db888e03a5da54ad83190109fbd0be2bf21f3d52728b2857ca9133a3c6d87d"
Jul 26 16:06:40.726411 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:40.726379    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"]
Jul 26 16:06:40.726886 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:40.726632    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g" podUID="024f6057-1134-4945-bc33-1798a39cf6b4" containerName="maas-api" containerID="cri-o://59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337" gracePeriod=30
Jul 26 16:06:40.766057 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:40.766019    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"]
Jul 26 16:06:40.766302 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:40.766275    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j" podUID="52f11902-4b34-4147-8d76-26d09af45dcb" containerName="payload-processing" containerID="cri-o://416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2" gracePeriod=30
Jul 26 16:06:40.771248 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:40.771222    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"]
Jul 26 16:06:40.771450 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:40.771410    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m" podUID="c3307aa3-c0ec-4a7e-85a0-12173a05332c" containerName="payload-pre-processing" containerID="cri-o://7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a" gracePeriod=30
Jul 26 16:06:41.034703 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.034679    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:41.038479 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.038455    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:41.073741 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.073674    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:41.093711 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.093676    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlzsl\" (UniqueName: \"kubernetes.io/projected/c3307aa3-c0ec-4a7e-85a0-12173a05332c-kube-api-access-qlzsl\") pod \"c3307aa3-c0ec-4a7e-85a0-12173a05332c\" (UID: \"c3307aa3-c0ec-4a7e-85a0-12173a05332c\") "
Jul 26 16:06:41.093711 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.093713    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-844b9\" (UniqueName: \"kubernetes.io/projected/024f6057-1134-4945-bc33-1798a39cf6b4-kube-api-access-844b9\") pod \"024f6057-1134-4945-bc33-1798a39cf6b4\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") "
Jul 26 16:06:41.093969 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.093776    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024f6057-1134-4945-bc33-1798a39cf6b4-openshift-service-ca\") pod \"024f6057-1134-4945-bc33-1798a39cf6b4\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") "
Jul 26 16:06:41.093969 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.093796    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3307aa3-c0ec-4a7e-85a0-12173a05332c-plugins-config-volume\") pod \"c3307aa3-c0ec-4a7e-85a0-12173a05332c\" (UID: \"c3307aa3-c0ec-4a7e-85a0-12173a05332c\") "
Jul 26 16:06:41.093969 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.093824    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024f6057-1134-4945-bc33-1798a39cf6b4-maas-api-tls\") pod \"024f6057-1134-4945-bc33-1798a39cf6b4\" (UID: \"024f6057-1134-4945-bc33-1798a39cf6b4\") "
Jul 26 16:06:41.094243 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.094205    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3307aa3-c0ec-4a7e-85a0-12173a05332c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c3307aa3-c0ec-4a7e-85a0-12173a05332c" (UID: "c3307aa3-c0ec-4a7e-85a0-12173a05332c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:06:41.094243 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.094216    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/024f6057-1134-4945-bc33-1798a39cf6b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "024f6057-1134-4945-bc33-1798a39cf6b4" (UID: "024f6057-1134-4945-bc33-1798a39cf6b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:06:41.095878 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.095856    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024f6057-1134-4945-bc33-1798a39cf6b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "024f6057-1134-4945-bc33-1798a39cf6b4" (UID: "024f6057-1134-4945-bc33-1798a39cf6b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:06:41.096029 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.096006    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3307aa3-c0ec-4a7e-85a0-12173a05332c-kube-api-access-qlzsl" (OuterVolumeSpecName: "kube-api-access-qlzsl") pod "c3307aa3-c0ec-4a7e-85a0-12173a05332c" (UID: "c3307aa3-c0ec-4a7e-85a0-12173a05332c"). InnerVolumeSpecName "kube-api-access-qlzsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:06:41.096078 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.096013    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/024f6057-1134-4945-bc33-1798a39cf6b4-kube-api-access-844b9" (OuterVolumeSpecName: "kube-api-access-844b9") pod "024f6057-1134-4945-bc33-1798a39cf6b4" (UID: "024f6057-1134-4945-bc33-1798a39cf6b4"). InnerVolumeSpecName "kube-api-access-844b9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:06:41.195261 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.195234    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slglj\" (UniqueName: \"kubernetes.io/projected/52f11902-4b34-4147-8d76-26d09af45dcb-kube-api-access-slglj\") pod \"52f11902-4b34-4147-8d76-26d09af45dcb\" (UID: \"52f11902-4b34-4147-8d76-26d09af45dcb\") "
Jul 26 16:06:41.195417 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.195278    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52f11902-4b34-4147-8d76-26d09af45dcb-plugins-config-volume\") pod \"52f11902-4b34-4147-8d76-26d09af45dcb\" (UID: \"52f11902-4b34-4147-8d76-26d09af45dcb\") "
Jul 26 16:06:41.195465 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.195445    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024f6057-1134-4945-bc33-1798a39cf6b4-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:06:41.195465 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.195455    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3307aa3-c0ec-4a7e-85a0-12173a05332c-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:06:41.195526 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.195464    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024f6057-1134-4945-bc33-1798a39cf6b4-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:06:41.195526 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.195474    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlzsl\" (UniqueName: \"kubernetes.io/projected/c3307aa3-c0ec-4a7e-85a0-12173a05332c-kube-api-access-qlzsl\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:06:41.195526 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.195482    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-844b9\" (UniqueName: \"kubernetes.io/projected/024f6057-1134-4945-bc33-1798a39cf6b4-kube-api-access-844b9\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:06:41.195639 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.195617    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52f11902-4b34-4147-8d76-26d09af45dcb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52f11902-4b34-4147-8d76-26d09af45dcb" (UID: "52f11902-4b34-4147-8d76-26d09af45dcb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:06:41.197268 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.197244    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f11902-4b34-4147-8d76-26d09af45dcb-kube-api-access-slglj" (OuterVolumeSpecName: "kube-api-access-slglj") pod "52f11902-4b34-4147-8d76-26d09af45dcb" (UID: "52f11902-4b34-4147-8d76-26d09af45dcb"). InnerVolumeSpecName "kube-api-access-slglj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:06:41.217644 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.217617    2560 generic.go:356] "Generic (PLEG): container finished" podID="c3307aa3-c0ec-4a7e-85a0-12173a05332c" containerID="7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a" exitCode=0
Jul 26 16:06:41.217786 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.217666    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"
Jul 26 16:06:41.217786 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.217707    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m" event={"ID":"c3307aa3-c0ec-4a7e-85a0-12173a05332c","Type":"ContainerDied","Data":"7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a"}
Jul 26 16:06:41.217786 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.217760    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m" event={"ID":"c3307aa3-c0ec-4a7e-85a0-12173a05332c","Type":"ContainerDied","Data":"00e9de22f7ca339076b2503c4d6ed1ac28185d8294cb34827bbc378b4e661dd7"}
Jul 26 16:06:41.217978 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.217785    2560 scope.go:117] "RemoveContainer" containerID="7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a"
Jul 26 16:06:41.219021 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.219001    2560 generic.go:356] "Generic (PLEG): container finished" podID="52f11902-4b34-4147-8d76-26d09af45dcb" containerID="416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2" exitCode=0
Jul 26 16:06:41.219110 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.219040    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"
Jul 26 16:06:41.219110 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.219090    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j" event={"ID":"52f11902-4b34-4147-8d76-26d09af45dcb","Type":"ContainerDied","Data":"416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2"}
Jul 26 16:06:41.219205 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.219129    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j" event={"ID":"52f11902-4b34-4147-8d76-26d09af45dcb","Type":"ContainerDied","Data":"543be14d046948e3b9ee20ec493623235a9cf2e7a248ebe6ff9be8dee7806860"}
Jul 26 16:06:41.220848 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.220826    2560 generic.go:356] "Generic (PLEG): container finished" podID="024f6057-1134-4945-bc33-1798a39cf6b4" containerID="59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337" exitCode=0
Jul 26 16:06:41.220981 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.220860    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g" event={"ID":"024f6057-1134-4945-bc33-1798a39cf6b4","Type":"ContainerDied","Data":"59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337"}
Jul 26 16:06:41.220981 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.220873    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"
Jul 26 16:06:41.221090 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.220998    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g" event={"ID":"024f6057-1134-4945-bc33-1798a39cf6b4","Type":"ContainerDied","Data":"d830357ff279aa30f10f4a24e43731042f2fb267b818d4489457492f548ddb34"}
Jul 26 16:06:41.227333 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.227313    2560 scope.go:117] "RemoveContainer" containerID="7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a"
Jul 26 16:06:41.227609 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:06:41.227588    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a\": container with ID starting with 7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a not found: ID does not exist" containerID="7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a"
Jul 26 16:06:41.227676 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.227616    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a"} err="failed to get container status \"7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a\": rpc error: code = NotFound desc = could not find container \"7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a\": container with ID starting with 7cc8f180980497374a2e8baa7075ef4b2b529df75b8a887b9227e77ea870e75a not found: ID does not exist"
Jul 26 16:06:41.227676 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.227633    2560 scope.go:117] "RemoveContainer" containerID="416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2"
Jul 26 16:06:41.232610 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.232564    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"]
Jul 26 16:06:41.235823 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.235805    2560 scope.go:117] "RemoveContainer" containerID="416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2"
Jul 26 16:06:41.236108 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:06:41.236091    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2\": container with ID starting with 416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2 not found: ID does not exist" containerID="416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2"
Jul 26 16:06:41.236176 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.236115    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2"} err="failed to get container status \"416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2\": rpc error: code = NotFound desc = could not find container \"416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2\": container with ID starting with 416f929b954fa2e167cb523503b6ab7f3c659271a2a7e845ad6a385f591736d2 not found: ID does not exist"
Jul 26 16:06:41.236176 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.236130    2560 scope.go:117] "RemoveContainer" containerID="59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337"
Jul 26 16:06:41.236645 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.236627    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c17951b0-79cdf7dc56-9544m"]
Jul 26 16:06:41.243690 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.243672    2560 scope.go:117] "RemoveContainer" containerID="59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337"
Jul 26 16:06:41.243956 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:06:41.243934    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337\": container with ID starting with 59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337 not found: ID does not exist" containerID="59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337"
Jul 26 16:06:41.244019 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.243965    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337"} err="failed to get container status \"59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337\": rpc error: code = NotFound desc = could not find container \"59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337\": container with ID starting with 59976c13f27cedbdf245c5c400c33f13ebe243fbaf9bed34af2776cca7b49337 not found: ID does not exist"
Jul 26 16:06:41.249292 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.249272    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"]
Jul 26 16:06:41.252834 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.252815    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c17951b0-6bd4467b6b-cfs6j"]
Jul 26 16:06:41.262438 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.262418    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"]
Jul 26 16:06:41.265805 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.265787    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c17951b0-59d66b47fd-tsq7g"]
Jul 26 16:06:41.296309 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.296279    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slglj\" (UniqueName: \"kubernetes.io/projected/52f11902-4b34-4147-8d76-26d09af45dcb-kube-api-access-slglj\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:06:41.296309 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:41.296305    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52f11902-4b34-4147-8d76-26d09af45dcb-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:06:43.138845 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:43.138812    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="024f6057-1134-4945-bc33-1798a39cf6b4" path="/var/lib/kubelet/pods/024f6057-1134-4945-bc33-1798a39cf6b4/volumes"
Jul 26 16:06:43.139239 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:43.139163    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52f11902-4b34-4147-8d76-26d09af45dcb" path="/var/lib/kubelet/pods/52f11902-4b34-4147-8d76-26d09af45dcb/volumes"
Jul 26 16:06:43.139464 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:43.139451    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3307aa3-c0ec-4a7e-85a0-12173a05332c" path="/var/lib/kubelet/pods/c3307aa3-c0ec-4a7e-85a0-12173a05332c/volumes"
Jul 26 16:06:45.404338 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:45.404301    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"]
Jul 26 16:06:45.406355 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:45.406329    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c17951b0-6ffhx"]
Jul 26 16:06:47.139161 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:06:47.139126    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c49cd25-d42e-4374-aa24-6fa341f2d7b3" path="/var/lib/kubelet/pods/0c49cd25-d42e-4374-aa24-6fa341f2d7b3/volumes"
Jul 26 16:08:25.859436 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859399    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"]
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859777    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52f11902-4b34-4147-8d76-26d09af45dcb" containerName="payload-processing"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859791    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="52f11902-4b34-4147-8d76-26d09af45dcb" containerName="payload-processing"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859806    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3307aa3-c0ec-4a7e-85a0-12173a05332c" containerName="payload-pre-processing"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859812    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3307aa3-c0ec-4a7e-85a0-12173a05332c" containerName="payload-pre-processing"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859824    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024f6057-1134-4945-bc33-1798a39cf6b4" containerName="maas-api"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859833    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="024f6057-1134-4945-bc33-1798a39cf6b4" containerName="maas-api"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859847    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c49cd25-d42e-4374-aa24-6fa341f2d7b3" containerName="revoke-keys"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859852    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c49cd25-d42e-4374-aa24-6fa341f2d7b3" containerName="revoke-keys"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859918    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="52f11902-4b34-4147-8d76-26d09af45dcb" containerName="payload-processing"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859929    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c49cd25-d42e-4374-aa24-6fa341f2d7b3" containerName="revoke-keys"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859934    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3307aa3-c0ec-4a7e-85a0-12173a05332c" containerName="payload-pre-processing"
Jul 26 16:08:25.860030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.859942    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="024f6057-1134-4945-bc33-1798a39cf6b4" containerName="maas-api"
Jul 26 16:08:25.863051 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.863031    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:25.865150 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.865127    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7827eee1-serving-cert\""
Jul 26 16:08:25.874263 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.873129    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"]
Jul 26 16:08:25.946967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.946931    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f27kh\" (UniqueName: \"kubernetes.io/projected/69417d08-934c-4238-b701-af7575df0449-kube-api-access-f27kh\") pod \"maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:25.947160 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.947009    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69417d08-934c-4238-b701-af7575df0449-openshift-service-ca\") pod \"maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:25.947160 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:25.947070    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69417d08-934c-4238-b701-af7575df0449-maas-api-tls\") pod \"maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:26.048053 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.048021    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f27kh\" (UniqueName: \"kubernetes.io/projected/69417d08-934c-4238-b701-af7575df0449-kube-api-access-f27kh\") pod \"maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:26.048387 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.048069    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69417d08-934c-4238-b701-af7575df0449-openshift-service-ca\") pod \"maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:26.048387 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.048099    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69417d08-934c-4238-b701-af7575df0449-maas-api-tls\") pod \"maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:26.048820 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.048796    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69417d08-934c-4238-b701-af7575df0449-openshift-service-ca\") pod \"maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:26.050326 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.050306    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69417d08-934c-4238-b701-af7575df0449-maas-api-tls\") pod \"maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:26.066024 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.065997    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f27kh\" (UniqueName: \"kubernetes.io/projected/69417d08-934c-4238-b701-af7575df0449-kube-api-access-f27kh\") pod \"maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:26.174141 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.174112    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:26.309805 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.309742    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"]
Jul 26 16:08:26.313062 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:08:26.313019    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69417d08_934c_4238_b701_af7575df0449.slice/crio-a12273b9712f8343dc682a52dc33a439d303cc9ad28601a2969f59135bf1479b WatchSource:0}: Error finding container a12273b9712f8343dc682a52dc33a439d303cc9ad28601a2969f59135bf1479b: Status 404 returned error can't find the container with id a12273b9712f8343dc682a52dc33a439d303cc9ad28601a2969f59135bf1479b
Jul 26 16:08:26.552485 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.552396    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"]
Jul 26 16:08:26.556987 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.556964    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:26.561177 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.561150    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7827eee1\""
Jul 26 16:08:26.561344 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.561202    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7827eee1-dockercfg-sfgff\""
Jul 26 16:08:26.572765 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.572733    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"]
Jul 26 16:08:26.585736 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.585695    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"]
Jul 26 16:08:26.588953 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.588931    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:26.599493 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.599424    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"]
Jul 26 16:08:26.653948 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.653881    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8820a2c2-44de-4692-ad08-376d90a94e49-plugins-config-volume\") pod \"payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk\" (UID: \"8820a2c2-44de-4692-ad08-376d90a94e49\") " pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:26.654167 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.653957    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rbbp\" (UniqueName: \"kubernetes.io/projected/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-kube-api-access-8rbbp\") pod \"payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw\" (UID: \"e6fc5d48-ac2a-424c-bebe-6859cd0cef22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:26.654167 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.653981    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlp26\" (UniqueName: \"kubernetes.io/projected/8820a2c2-44de-4692-ad08-376d90a94e49-kube-api-access-tlp26\") pod \"payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk\" (UID: \"8820a2c2-44de-4692-ad08-376d90a94e49\") " pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:26.654167 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.654026    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw\" (UID: \"e6fc5d48-ac2a-424c-bebe-6859cd0cef22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:26.696636 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.696604    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt" event={"ID":"69417d08-934c-4238-b701-af7575df0449","Type":"ContainerStarted","Data":"a12273b9712f8343dc682a52dc33a439d303cc9ad28601a2969f59135bf1479b"}
Jul 26 16:08:26.754594 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.754568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8820a2c2-44de-4692-ad08-376d90a94e49-plugins-config-volume\") pod \"payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk\" (UID: \"8820a2c2-44de-4692-ad08-376d90a94e49\") " pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:26.754763 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.754608    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rbbp\" (UniqueName: \"kubernetes.io/projected/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-kube-api-access-8rbbp\") pod \"payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw\" (UID: \"e6fc5d48-ac2a-424c-bebe-6859cd0cef22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:26.754763 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.754630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlp26\" (UniqueName: \"kubernetes.io/projected/8820a2c2-44de-4692-ad08-376d90a94e49-kube-api-access-tlp26\") pod \"payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk\" (UID: \"8820a2c2-44de-4692-ad08-376d90a94e49\") " pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:26.754763 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.754692    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw\" (UID: \"e6fc5d48-ac2a-424c-bebe-6859cd0cef22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:26.755551 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.755488    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw\" (UID: \"e6fc5d48-ac2a-424c-bebe-6859cd0cef22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:26.756005 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.755976    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8820a2c2-44de-4692-ad08-376d90a94e49-plugins-config-volume\") pod \"payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk\" (UID: \"8820a2c2-44de-4692-ad08-376d90a94e49\") " pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:26.762604 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.762577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlp26\" (UniqueName: \"kubernetes.io/projected/8820a2c2-44de-4692-ad08-376d90a94e49-kube-api-access-tlp26\") pod \"payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk\" (UID: \"8820a2c2-44de-4692-ad08-376d90a94e49\") " pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:26.762699 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.762630    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rbbp\" (UniqueName: \"kubernetes.io/projected/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-kube-api-access-8rbbp\") pod \"payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw\" (UID: \"e6fc5d48-ac2a-424c-bebe-6859cd0cef22\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:26.868045 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.867949    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:26.901671 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:26.901640    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:27.012634 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.012583    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"]
Jul 26 16:08:27.018255 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:08:27.018228    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6fc5d48_ac2a_424c_bebe_6859cd0cef22.slice/crio-cda04ddebc8b2e92b3d8a7c9ba92c817db372e08de198d557f3b42e6fcbeb99d WatchSource:0}: Error finding container cda04ddebc8b2e92b3d8a7c9ba92c817db372e08de198d557f3b42e6fcbeb99d: Status 404 returned error can't find the container with id cda04ddebc8b2e92b3d8a7c9ba92c817db372e08de198d557f3b42e6fcbeb99d
Jul 26 16:08:27.053747 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.053667    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"]
Jul 26 16:08:27.056931 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:08:27.056886    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8820a2c2_44de_4692_ad08_376d90a94e49.slice/crio-4db86aa6037d66b0fddf214babe37e13f0e958406e1baa88ee0b7b1350a12e4a WatchSource:0}: Error finding container 4db86aa6037d66b0fddf214babe37e13f0e958406e1baa88ee0b7b1350a12e4a: Status 404 returned error can't find the container with id 4db86aa6037d66b0fddf214babe37e13f0e958406e1baa88ee0b7b1350a12e4a
Jul 26 16:08:27.702120 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.702083    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw" event={"ID":"e6fc5d48-ac2a-424c-bebe-6859cd0cef22","Type":"ContainerStarted","Data":"18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d"}
Jul 26 16:08:27.702331 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.702145    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw" event={"ID":"e6fc5d48-ac2a-424c-bebe-6859cd0cef22","Type":"ContainerStarted","Data":"cda04ddebc8b2e92b3d8a7c9ba92c817db372e08de198d557f3b42e6fcbeb99d"}
Jul 26 16:08:27.702331 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.702171    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:27.703789 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.703762    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk" event={"ID":"8820a2c2-44de-4692-ad08-376d90a94e49","Type":"ContainerStarted","Data":"5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460"}
Jul 26 16:08:27.703914 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.703807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk" event={"ID":"8820a2c2-44de-4692-ad08-376d90a94e49","Type":"ContainerStarted","Data":"4db86aa6037d66b0fddf214babe37e13f0e958406e1baa88ee0b7b1350a12e4a"}
Jul 26 16:08:27.703914 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.703846    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:27.705210 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.705188    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt" event={"ID":"69417d08-934c-4238-b701-af7575df0449","Type":"ContainerStarted","Data":"3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff"}
Jul 26 16:08:27.705317 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.705303    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:27.719013 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.718961    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw" podStartSLOduration=1.718941451 podStartE2EDuration="1.718941451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:27.716479106 +0000 UTC m=+1823.215868951" watchObservedRunningTime="2026-07-26 16:08:27.718941451 +0000 UTC m=+1823.218331293"
Jul 26 16:08:27.732828 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.732785    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt" podStartSLOduration=2.356646382 podStartE2EDuration="2.732769235s" totalImagesPullingTime="376.122853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:27.732155132 +0000 UTC m=+1823.231544971" watchObservedRunningTime="2026-07-26 16:08:27.732769235 +0000 UTC m=+1823.232159075"
Jul 26 16:08:27.748627 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:27.748583    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk" podStartSLOduration=1.748569413 podStartE2EDuration="1.748569413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:08:27.746379486 +0000 UTC m=+1823.245769323" watchObservedRunningTime="2026-07-26 16:08:27.748569413 +0000 UTC m=+1823.247959250"
Jul 26 16:08:33.717122 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:33.717086    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:37.301503 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:37.301468    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"]
Jul 26 16:08:37.305099 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:37.305082    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"
Jul 26 16:08:37.307219 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:37.307200    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:08:37.313510 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:37.313489    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"]
Jul 26 16:08:37.346147 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:37.346121    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ddf4c43-76b9-440d-b994-fb7a64b60b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9\" (UID: \"7ddf4c43-76b9-440d-b994-fb7a64b60b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"
Jul 26 16:08:37.447167 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:37.447141    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ddf4c43-76b9-440d-b994-fb7a64b60b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9\" (UID: \"7ddf4c43-76b9-440d-b994-fb7a64b60b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"
Jul 26 16:08:37.447691 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:37.447674    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ddf4c43-76b9-440d-b994-fb7a64b60b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9\" (UID: \"7ddf4c43-76b9-440d-b994-fb7a64b60b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"
Jul 26 16:08:37.616551 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:37.616493    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"
Jul 26 16:08:37.747823 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:37.747648    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"]
Jul 26 16:08:37.750789 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:08:37.750765    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ddf4c43_76b9_440d_b994_fb7a64b60b0c.slice/crio-b425628e543c0950a5c89bb373cf4e2a8a718becc5f634d6c30d03f1dd3563c0 WatchSource:0}: Error finding container b425628e543c0950a5c89bb373cf4e2a8a718becc5f634d6c30d03f1dd3563c0: Status 404 returned error can't find the container with id b425628e543c0950a5c89bb373cf4e2a8a718becc5f634d6c30d03f1dd3563c0
Jul 26 16:08:37.757407 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:37.757381    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9" event={"ID":"7ddf4c43-76b9-440d-b994-fb7a64b60b0c","Type":"ContainerStarted","Data":"b425628e543c0950a5c89bb373cf4e2a8a718becc5f634d6c30d03f1dd3563c0"}
Jul 26 16:08:38.712934 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:38.712821    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw" podUID="e6fc5d48-ac2a-424c-bebe-6859cd0cef22" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.51:9004: connect: connection refused"
Jul 26 16:08:38.713420 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:38.713399    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:38.764500 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:38.764468    2560 generic.go:356] "Generic (PLEG): container finished" podID="7ddf4c43-76b9-440d-b994-fb7a64b60b0c" containerID="7c3f1d753f6b1b74c1e9dfd9fe5861bc06d74a368efd8d36947006bbd9ed0a06" exitCode=0
Jul 26 16:08:38.764644 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:38.764507    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9" event={"ID":"7ddf4c43-76b9-440d-b994-fb7a64b60b0c","Type":"ContainerDied","Data":"7c3f1d753f6b1b74c1e9dfd9fe5861bc06d74a368efd8d36947006bbd9ed0a06"}
Jul 26 16:08:39.898417 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:39.898396    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"
Jul 26 16:08:39.966388 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:39.966362    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ddf4c43-76b9-440d-b994-fb7a64b60b0c-maas-api-service-ca\") pod \"7ddf4c43-76b9-440d-b994-fb7a64b60b0c\" (UID: \"7ddf4c43-76b9-440d-b994-fb7a64b60b0c\") "
Jul 26 16:08:39.966670 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:39.966649    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ddf4c43-76b9-440d-b994-fb7a64b60b0c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ddf4c43-76b9-440d-b994-fb7a64b60b0c" (UID: "7ddf4c43-76b9-440d-b994-fb7a64b60b0c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:08:40.066995 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:40.066929    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ddf4c43-76b9-440d-b994-fb7a64b60b0c-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:08:40.774580 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:40.774547    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"
Jul 26 16:08:40.774737 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:40.774599    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9" event={"ID":"7ddf4c43-76b9-440d-b994-fb7a64b60b0c","Type":"ContainerDied","Data":"b425628e543c0950a5c89bb373cf4e2a8a718becc5f634d6c30d03f1dd3563c0"}
Jul 26 16:08:40.774737 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:40.774643    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b425628e543c0950a5c89bb373cf4e2a8a718becc5f634d6c30d03f1dd3563c0"
Jul 26 16:08:47.766119 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:47.766085    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"]
Jul 26 16:08:47.766720 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:47.766322    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt" podUID="69417d08-934c-4238-b701-af7575df0449" containerName="maas-api" containerID="cri-o://3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff" gracePeriod=30
Jul 26 16:08:47.816475 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:47.816440    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"]
Jul 26 16:08:47.816701 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:47.816673    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk" podUID="8820a2c2-44de-4692-ad08-376d90a94e49" containerName="payload-processing" containerID="cri-o://5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460" gracePeriod=30
Jul 26 16:08:47.822249 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:47.822226    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"]
Jul 26 16:08:47.822546 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:47.822397    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw" podUID="e6fc5d48-ac2a-424c-bebe-6859cd0cef22" containerName="payload-pre-processing" containerID="cri-o://18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d" gracePeriod=30
Jul 26 16:08:47.822990 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:47.822962    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:48.072330 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.072298    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:48.097713 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.097683    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:48.107228 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.107208    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:48.130790 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.130766    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69417d08-934c-4238-b701-af7575df0449-openshift-service-ca\") pod \"69417d08-934c-4238-b701-af7575df0449\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") "
Jul 26 16:08:48.130973 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.130851    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f27kh\" (UniqueName: \"kubernetes.io/projected/69417d08-934c-4238-b701-af7575df0449-kube-api-access-f27kh\") pod \"69417d08-934c-4238-b701-af7575df0449\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") "
Jul 26 16:08:48.130973 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.130954    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69417d08-934c-4238-b701-af7575df0449-maas-api-tls\") pod \"69417d08-934c-4238-b701-af7575df0449\" (UID: \"69417d08-934c-4238-b701-af7575df0449\") "
Jul 26 16:08:48.131149 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.131127    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69417d08-934c-4238-b701-af7575df0449-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "69417d08-934c-4238-b701-af7575df0449" (UID: "69417d08-934c-4238-b701-af7575df0449"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:08:48.131200 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.131186    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69417d08-934c-4238-b701-af7575df0449-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:08:48.132969 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.132948    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69417d08-934c-4238-b701-af7575df0449-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "69417d08-934c-4238-b701-af7575df0449" (UID: "69417d08-934c-4238-b701-af7575df0449"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:08:48.133063 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.132962    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69417d08-934c-4238-b701-af7575df0449-kube-api-access-f27kh" (OuterVolumeSpecName: "kube-api-access-f27kh") pod "69417d08-934c-4238-b701-af7575df0449" (UID: "69417d08-934c-4238-b701-af7575df0449"). InnerVolumeSpecName "kube-api-access-f27kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:08:48.231816 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.231782    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlp26\" (UniqueName: \"kubernetes.io/projected/8820a2c2-44de-4692-ad08-376d90a94e49-kube-api-access-tlp26\") pod \"8820a2c2-44de-4692-ad08-376d90a94e49\" (UID: \"8820a2c2-44de-4692-ad08-376d90a94e49\") "
Jul 26 16:08:48.232002 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.231843    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-plugins-config-volume\") pod \"e6fc5d48-ac2a-424c-bebe-6859cd0cef22\" (UID: \"e6fc5d48-ac2a-424c-bebe-6859cd0cef22\") "
Jul 26 16:08:48.232002 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.231868    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rbbp\" (UniqueName: \"kubernetes.io/projected/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-kube-api-access-8rbbp\") pod \"e6fc5d48-ac2a-424c-bebe-6859cd0cef22\" (UID: \"e6fc5d48-ac2a-424c-bebe-6859cd0cef22\") "
Jul 26 16:08:48.232002 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.231886    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8820a2c2-44de-4692-ad08-376d90a94e49-plugins-config-volume\") pod \"8820a2c2-44de-4692-ad08-376d90a94e49\" (UID: \"8820a2c2-44de-4692-ad08-376d90a94e49\") "
Jul 26 16:08:48.232146 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.232037    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69417d08-934c-4238-b701-af7575df0449-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:08:48.232146 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.232047    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f27kh\" (UniqueName: \"kubernetes.io/projected/69417d08-934c-4238-b701-af7575df0449-kube-api-access-f27kh\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:08:48.232318 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.232289    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6fc5d48-ac2a-424c-bebe-6859cd0cef22" (UID: "e6fc5d48-ac2a-424c-bebe-6859cd0cef22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:08:48.232375 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.232298    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8820a2c2-44de-4692-ad08-376d90a94e49-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8820a2c2-44de-4692-ad08-376d90a94e49" (UID: "8820a2c2-44de-4692-ad08-376d90a94e49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:08:48.233945 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.233924    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8820a2c2-44de-4692-ad08-376d90a94e49-kube-api-access-tlp26" (OuterVolumeSpecName: "kube-api-access-tlp26") pod "8820a2c2-44de-4692-ad08-376d90a94e49" (UID: "8820a2c2-44de-4692-ad08-376d90a94e49"). InnerVolumeSpecName "kube-api-access-tlp26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:08:48.233945 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.233921    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-kube-api-access-8rbbp" (OuterVolumeSpecName: "kube-api-access-8rbbp") pod "e6fc5d48-ac2a-424c-bebe-6859cd0cef22" (UID: "e6fc5d48-ac2a-424c-bebe-6859cd0cef22"). InnerVolumeSpecName "kube-api-access-8rbbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:08:48.333350 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.333282    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlp26\" (UniqueName: \"kubernetes.io/projected/8820a2c2-44de-4692-ad08-376d90a94e49-kube-api-access-tlp26\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:08:48.333350 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.333305    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:08:48.333350 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.333315    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rbbp\" (UniqueName: \"kubernetes.io/projected/e6fc5d48-ac2a-424c-bebe-6859cd0cef22-kube-api-access-8rbbp\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:08:48.333350 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.333323    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8820a2c2-44de-4692-ad08-376d90a94e49-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:08:48.814019 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.813984    2560 generic.go:356] "Generic (PLEG): container finished" podID="69417d08-934c-4238-b701-af7575df0449" containerID="3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff" exitCode=0
Jul 26 16:08:48.814471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.814032    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"
Jul 26 16:08:48.814471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.814063    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt" event={"ID":"69417d08-934c-4238-b701-af7575df0449","Type":"ContainerDied","Data":"3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff"}
Jul 26 16:08:48.814471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.814111    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt" event={"ID":"69417d08-934c-4238-b701-af7575df0449","Type":"ContainerDied","Data":"a12273b9712f8343dc682a52dc33a439d303cc9ad28601a2969f59135bf1479b"}
Jul 26 16:08:48.814471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.814137    2560 scope.go:117] "RemoveContainer" containerID="3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff"
Jul 26 16:08:48.815624 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.815323    2560 generic.go:356] "Generic (PLEG): container finished" podID="e6fc5d48-ac2a-424c-bebe-6859cd0cef22" containerID="18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d" exitCode=0
Jul 26 16:08:48.815624 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.815359    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"
Jul 26 16:08:48.815624 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.815413    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw" event={"ID":"e6fc5d48-ac2a-424c-bebe-6859cd0cef22","Type":"ContainerDied","Data":"18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d"}
Jul 26 16:08:48.815624 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.815466    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw" event={"ID":"e6fc5d48-ac2a-424c-bebe-6859cd0cef22","Type":"ContainerDied","Data":"cda04ddebc8b2e92b3d8a7c9ba92c817db372e08de198d557f3b42e6fcbeb99d"}
Jul 26 16:08:48.816926 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.816886    2560 generic.go:356] "Generic (PLEG): container finished" podID="8820a2c2-44de-4692-ad08-376d90a94e49" containerID="5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460" exitCode=0
Jul 26 16:08:48.817051 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.816954    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"
Jul 26 16:08:48.817051 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.816970    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk" event={"ID":"8820a2c2-44de-4692-ad08-376d90a94e49","Type":"ContainerDied","Data":"5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460"}
Jul 26 16:08:48.817051 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.817004    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk" event={"ID":"8820a2c2-44de-4692-ad08-376d90a94e49","Type":"ContainerDied","Data":"4db86aa6037d66b0fddf214babe37e13f0e958406e1baa88ee0b7b1350a12e4a"}
Jul 26 16:08:48.832634 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.832615    2560 scope.go:117] "RemoveContainer" containerID="3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff"
Jul 26 16:08:48.832884 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:08:48.832863    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff\": container with ID starting with 3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff not found: ID does not exist" containerID="3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff"
Jul 26 16:08:48.832967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.832912    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff"} err="failed to get container status \"3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff\": rpc error: code = NotFound desc = could not find container \"3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff\": container with ID starting with 3ae517743162aa83b826d2495da4432ad0faa8358210b27cb7cddb5c3a664eff not found: ID does not exist"
Jul 26 16:08:48.832967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.832930    2560 scope.go:117] "RemoveContainer" containerID="18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d"
Jul 26 16:08:48.840477 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.840436    2560 scope.go:117] "RemoveContainer" containerID="18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d"
Jul 26 16:08:48.840809 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:08:48.840780    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d\": container with ID starting with 18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d not found: ID does not exist" containerID="18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d"
Jul 26 16:08:48.840922 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.840837    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d"} err="failed to get container status \"18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d\": rpc error: code = NotFound desc = could not find container \"18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d\": container with ID starting with 18148950c129ba54c247809eccdd061b1339049b2bd99d2198f597f075bca98d not found: ID does not exist"
Jul 26 16:08:48.840922 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.840859    2560 scope.go:117] "RemoveContainer" containerID="5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460"
Jul 26 16:08:48.842652 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.842632    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"]
Jul 26 16:08:48.845736 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.845716    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7827eee1-fb68fc9dd-qd9dt"]
Jul 26 16:08:48.851037 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.851006    2560 scope.go:117] "RemoveContainer" containerID="5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460"
Jul 26 16:08:48.851301 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:08:48.851278    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460\": container with ID starting with 5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460 not found: ID does not exist" containerID="5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460"
Jul 26 16:08:48.851365 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.851308    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460"} err="failed to get container status \"5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460\": rpc error: code = NotFound desc = could not find container \"5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460\": container with ID starting with 5bf5d431ddb1401e7b0ac75c13d7108da2ab719a151bdde795d52bd2d9f8c460 not found: ID does not exist"
Jul 26 16:08:48.854807 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.854783    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"]
Jul 26 16:08:48.858966 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.858947    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7827eee1-5b77445fd-6bdpk"]
Jul 26 16:08:48.868352 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.868332    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"]
Jul 26 16:08:48.872039 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:48.872017    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7827eee1-7f4bc6694d-f4fqw"]
Jul 26 16:08:49.138774 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:49.138746    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69417d08-934c-4238-b701-af7575df0449" path="/var/lib/kubelet/pods/69417d08-934c-4238-b701-af7575df0449/volumes"
Jul 26 16:08:49.139132 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:49.139118    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8820a2c2-44de-4692-ad08-376d90a94e49" path="/var/lib/kubelet/pods/8820a2c2-44de-4692-ad08-376d90a94e49/volumes"
Jul 26 16:08:49.139427 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:49.139415    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6fc5d48-ac2a-424c-bebe-6859cd0cef22" path="/var/lib/kubelet/pods/e6fc5d48-ac2a-424c-bebe-6859cd0cef22/volumes"
Jul 26 16:08:52.368046 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:52.368010    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"]
Jul 26 16:08:52.370532 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:52.370508    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7827eee1-2jsg9"]
Jul 26 16:08:53.138224 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:08:53.138193    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ddf4c43-76b9-440d-b994-fb7a64b60b0c" path="/var/lib/kubelet/pods/7ddf4c43-76b9-440d-b994-fb7a64b60b0c/volumes"
Jul 26 16:09:21.217276 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217242    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"]
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217557    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69417d08-934c-4238-b701-af7575df0449" containerName="maas-api"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217567    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="69417d08-934c-4238-b701-af7575df0449" containerName="maas-api"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217582    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8820a2c2-44de-4692-ad08-376d90a94e49" containerName="payload-processing"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217588    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8820a2c2-44de-4692-ad08-376d90a94e49" containerName="payload-processing"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217596    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6fc5d48-ac2a-424c-bebe-6859cd0cef22" containerName="payload-pre-processing"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217602    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6fc5d48-ac2a-424c-bebe-6859cd0cef22" containerName="payload-pre-processing"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217614    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ddf4c43-76b9-440d-b994-fb7a64b60b0c" containerName="revoke-keys"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217619    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ddf4c43-76b9-440d-b994-fb7a64b60b0c" containerName="revoke-keys"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217673    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6fc5d48-ac2a-424c-bebe-6859cd0cef22" containerName="payload-pre-processing"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217683    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8820a2c2-44de-4692-ad08-376d90a94e49" containerName="payload-processing"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217689    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="69417d08-934c-4238-b701-af7575df0449" containerName="maas-api"
Jul 26 16:09:21.218065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.217696    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ddf4c43-76b9-440d-b994-fb7a64b60b0c" containerName="revoke-keys"
Jul 26 16:09:21.220598 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.220579    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.222601 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.222576    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1a44779b-serving-cert\""
Jul 26 16:09:21.231760 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.231738    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"]
Jul 26 16:09:21.306256 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.306219    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a5b868-4220-4d33-87cd-c44662751d2f-openshift-service-ca\") pod \"maas-api-e2e-mt-1a44779b-79b4894999-lfxrr\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.306459 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.306340    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2zqn\" (UniqueName: \"kubernetes.io/projected/91a5b868-4220-4d33-87cd-c44662751d2f-kube-api-access-r2zqn\") pod \"maas-api-e2e-mt-1a44779b-79b4894999-lfxrr\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.306459 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.306422    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a5b868-4220-4d33-87cd-c44662751d2f-maas-api-tls\") pod \"maas-api-e2e-mt-1a44779b-79b4894999-lfxrr\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.407904 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.407870    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a5b868-4220-4d33-87cd-c44662751d2f-maas-api-tls\") pod \"maas-api-e2e-mt-1a44779b-79b4894999-lfxrr\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.408101 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.407998    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a5b868-4220-4d33-87cd-c44662751d2f-openshift-service-ca\") pod \"maas-api-e2e-mt-1a44779b-79b4894999-lfxrr\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.408101 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.408040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2zqn\" (UniqueName: \"kubernetes.io/projected/91a5b868-4220-4d33-87cd-c44662751d2f-kube-api-access-r2zqn\") pod \"maas-api-e2e-mt-1a44779b-79b4894999-lfxrr\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.408617 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.408591    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a5b868-4220-4d33-87cd-c44662751d2f-openshift-service-ca\") pod \"maas-api-e2e-mt-1a44779b-79b4894999-lfxrr\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.410371 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.410344    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a5b868-4220-4d33-87cd-c44662751d2f-maas-api-tls\") pod \"maas-api-e2e-mt-1a44779b-79b4894999-lfxrr\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.418252 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.418229    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2zqn\" (UniqueName: \"kubernetes.io/projected/91a5b868-4220-4d33-87cd-c44662751d2f-kube-api-access-r2zqn\") pod \"maas-api-e2e-mt-1a44779b-79b4894999-lfxrr\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.531430 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.531362    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:21.671803 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.671774    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"]
Jul 26 16:09:21.674162 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:09:21.674135    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a5b868_4220_4d33_87cd_c44662751d2f.slice/crio-5c76a8c339e21e906ca1a4a47759cad6dff94eada20c042b744112455bf8e4ed WatchSource:0}: Error finding container 5c76a8c339e21e906ca1a4a47759cad6dff94eada20c042b744112455bf8e4ed: Status 404 returned error can't find the container with id 5c76a8c339e21e906ca1a4a47759cad6dff94eada20c042b744112455bf8e4ed
Jul 26 16:09:21.675542 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.675519    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:09:21.906810 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.906770    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"]
Jul 26 16:09:21.911204 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.911178    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:09:21.914614 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.914592    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-1a44779b-dockercfg-wkxlr\""
Jul 26 16:09:21.915116 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.915075    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-1a44779b\""
Jul 26 16:09:21.924145 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.924122    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"]
Jul 26 16:09:21.938235 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.938213    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"]
Jul 26 16:09:21.941293 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.941275    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:09:21.950469 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.950444    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"]
Jul 26 16:09:21.975072 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:21.975043    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr" event={"ID":"91a5b868-4220-4d33-87cd-c44662751d2f","Type":"ContainerStarted","Data":"5c76a8c339e21e906ca1a4a47759cad6dff94eada20c042b744112455bf8e4ed"}
Jul 26 16:09:22.014067 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.014029    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1fcea56-d53b-42da-82ee-650eb7068d1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz\" (UID: \"d1fcea56-d53b-42da-82ee-650eb7068d1e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:09:22.014369 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.014347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rnf9\" (UniqueName: \"kubernetes.io/projected/d1fcea56-d53b-42da-82ee-650eb7068d1e-kube-api-access-9rnf9\") pod \"payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz\" (UID: \"d1fcea56-d53b-42da-82ee-650eb7068d1e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:09:22.014536 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.014514    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-plugins-config-volume\") pod \"payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg\" (UID: \"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47\") " pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:09:22.014662 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.014600    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmbpw\" (UniqueName: \"kubernetes.io/projected/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-kube-api-access-zmbpw\") pod \"payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg\" (UID: \"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47\") " pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:09:22.115856 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.115818    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1fcea56-d53b-42da-82ee-650eb7068d1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz\" (UID: \"d1fcea56-d53b-42da-82ee-650eb7068d1e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:09:22.116054 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.115880    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rnf9\" (UniqueName: \"kubernetes.io/projected/d1fcea56-d53b-42da-82ee-650eb7068d1e-kube-api-access-9rnf9\") pod \"payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz\" (UID: \"d1fcea56-d53b-42da-82ee-650eb7068d1e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:09:22.116054 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.115944    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-plugins-config-volume\") pod \"payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg\" (UID: \"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47\") " pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:09:22.116054 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.116021    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmbpw\" (UniqueName: \"kubernetes.io/projected/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-kube-api-access-zmbpw\") pod \"payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg\" (UID: \"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47\") " pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:09:22.116777 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.116750    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1fcea56-d53b-42da-82ee-650eb7068d1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz\" (UID: \"d1fcea56-d53b-42da-82ee-650eb7068d1e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:09:22.117072 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.116976    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-plugins-config-volume\") pod \"payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg\" (UID: \"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47\") " pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:09:22.123479 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.123456    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rnf9\" (UniqueName: \"kubernetes.io/projected/d1fcea56-d53b-42da-82ee-650eb7068d1e-kube-api-access-9rnf9\") pod \"payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz\" (UID: \"d1fcea56-d53b-42da-82ee-650eb7068d1e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:09:22.123990 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.123970    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmbpw\" (UniqueName: \"kubernetes.io/projected/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-kube-api-access-zmbpw\") pod \"payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg\" (UID: \"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47\") " pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:09:22.222421 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.222332    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:09:22.251754 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.251716    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:09:22.373114 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.373018    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"]
Jul 26 16:09:22.375236 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:09:22.375185    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1fcea56_d53b_42da_82ee_650eb7068d1e.slice/crio-b4309316c884cf463756b44f51a9740f85551810f1db2fb993e3a0048a3a53ff WatchSource:0}: Error finding container b4309316c884cf463756b44f51a9740f85551810f1db2fb993e3a0048a3a53ff: Status 404 returned error can't find the container with id b4309316c884cf463756b44f51a9740f85551810f1db2fb993e3a0048a3a53ff
Jul 26 16:09:22.400058 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.399952    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"]
Jul 26 16:09:22.407048 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:09:22.407025    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e58e4c_04ba_41a9_9af3_ecc46bf8ee47.slice/crio-e84377955864a173668aebb82ae1f47e933b6e193735adc323b7aa5b7d9ca16b WatchSource:0}: Error finding container e84377955864a173668aebb82ae1f47e933b6e193735adc323b7aa5b7d9ca16b: Status 404 returned error can't find the container with id e84377955864a173668aebb82ae1f47e933b6e193735adc323b7aa5b7d9ca16b
Jul 26 16:09:22.982466 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.982433    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz" event={"ID":"d1fcea56-d53b-42da-82ee-650eb7068d1e","Type":"ContainerStarted","Data":"e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36"}
Jul 26 16:09:22.982661 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.982493    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz" event={"ID":"d1fcea56-d53b-42da-82ee-650eb7068d1e","Type":"ContainerStarted","Data":"b4309316c884cf463756b44f51a9740f85551810f1db2fb993e3a0048a3a53ff"}
Jul 26 16:09:22.982661 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.982525    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:09:22.984120 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.984089    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr" event={"ID":"91a5b868-4220-4d33-87cd-c44662751d2f","Type":"ContainerStarted","Data":"523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962"}
Jul 26 16:09:22.984243 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.984168    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:22.985620 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.985592    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg" event={"ID":"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47","Type":"ContainerStarted","Data":"c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9"}
Jul 26 16:09:22.985740 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.985632    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg" event={"ID":"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47","Type":"ContainerStarted","Data":"e84377955864a173668aebb82ae1f47e933b6e193735adc323b7aa5b7d9ca16b"}
Jul 26 16:09:22.985740 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.985680    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:09:22.998910 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:22.998849    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz" podStartSLOduration=1.99883175 podStartE2EDuration="1.99883175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:22.99711615 +0000 UTC m=+1878.496505987" watchObservedRunningTime="2026-07-26 16:09:22.99883175 +0000 UTC m=+1878.498221590"
Jul 26 16:09:23.013475 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:23.013427    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg" podStartSLOduration=2.013410104 podStartE2EDuration="2.013410104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:23.012080257 +0000 UTC m=+1878.511470097" watchObservedRunningTime="2026-07-26 16:09:23.013410104 +0000 UTC m=+1878.512799944"
Jul 26 16:09:23.028712 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:23.028662    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr" podStartSLOduration=1.669393705 podStartE2EDuration="2.028644288s" totalImagesPullingTime="359.250583ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:23.027557864 +0000 UTC m=+1878.526947698" watchObservedRunningTime="2026-07-26 16:09:23.028644288 +0000 UTC m=+1878.528034128"
Jul 26 16:09:28.995961 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:28.995866    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:09:33.992760 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:33.992730    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:09:33.993241 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:09:33.992800    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:10:00.002770 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:00.002725    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"]
Jul 26 16:10:00.007022 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:00.007001    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"
Jul 26 16:10:00.009032 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:00.009009    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:10:00.011588 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:00.011550    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"]
Jul 26 16:10:00.153332 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:00.153303    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5effbd5-2546-43ae-b605-4cb13eff02cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w\" (UID: \"d5effbd5-2546-43ae-b605-4cb13eff02cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"
Jul 26 16:10:00.253949 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:00.253858    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5effbd5-2546-43ae-b605-4cb13eff02cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w\" (UID: \"d5effbd5-2546-43ae-b605-4cb13eff02cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"
Jul 26 16:10:00.254475 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:00.254453    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5effbd5-2546-43ae-b605-4cb13eff02cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w\" (UID: \"d5effbd5-2546-43ae-b605-4cb13eff02cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"
Jul 26 16:10:00.319537 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:00.319509    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"
Jul 26 16:10:00.451420 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:00.451390    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"]
Jul 26 16:10:00.454646 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:10:00.454621    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5effbd5_2546_43ae_b605_4cb13eff02cc.slice/crio-ddc4c7433e36d2e7717ef5302f8deede77f8fdd418bb256cf9c4b5705c11fe5a WatchSource:0}: Error finding container ddc4c7433e36d2e7717ef5302f8deede77f8fdd418bb256cf9c4b5705c11fe5a: Status 404 returned error can't find the container with id ddc4c7433e36d2e7717ef5302f8deede77f8fdd418bb256cf9c4b5705c11fe5a
Jul 26 16:10:01.157210 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:01.157179    2560 generic.go:356] "Generic (PLEG): container finished" podID="d5effbd5-2546-43ae-b605-4cb13eff02cc" containerID="48bfebd468a14c1f9921d228d32e6283d241996ae2eee22113d0e3dc1186c678" exitCode=0
Jul 26 16:10:01.157560 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:01.157249    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w" event={"ID":"d5effbd5-2546-43ae-b605-4cb13eff02cc","Type":"ContainerDied","Data":"48bfebd468a14c1f9921d228d32e6283d241996ae2eee22113d0e3dc1186c678"}
Jul 26 16:10:01.157560 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:01.157294    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w" event={"ID":"d5effbd5-2546-43ae-b605-4cb13eff02cc","Type":"ContainerStarted","Data":"ddc4c7433e36d2e7717ef5302f8deede77f8fdd418bb256cf9c4b5705c11fe5a"}
Jul 26 16:10:02.289655 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:02.289630    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"
Jul 26 16:10:02.365822 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:02.365798    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5effbd5-2546-43ae-b605-4cb13eff02cc-maas-api-service-ca\") pod \"d5effbd5-2546-43ae-b605-4cb13eff02cc\" (UID: \"d5effbd5-2546-43ae-b605-4cb13eff02cc\") "
Jul 26 16:10:02.366148 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:02.366122    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5effbd5-2546-43ae-b605-4cb13eff02cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d5effbd5-2546-43ae-b605-4cb13eff02cc" (UID: "d5effbd5-2546-43ae-b605-4cb13eff02cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:10:02.466966 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:02.466862    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5effbd5-2546-43ae-b605-4cb13eff02cc-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:10:03.167845 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:03.167822    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"
Jul 26 16:10:03.168003 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:03.167817    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w" event={"ID":"d5effbd5-2546-43ae-b605-4cb13eff02cc","Type":"ContainerDied","Data":"ddc4c7433e36d2e7717ef5302f8deede77f8fdd418bb256cf9c4b5705c11fe5a"}
Jul 26 16:10:03.168003 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:03.167949    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddc4c7433e36d2e7717ef5302f8deede77f8fdd418bb256cf9c4b5705c11fe5a"
Jul 26 16:10:15.501306 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.501274    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"]
Jul 26 16:10:15.503803 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.501493    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr" podUID="91a5b868-4220-4d33-87cd-c44662751d2f" containerName="maas-api" containerID="cri-o://523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962" gracePeriod=30
Jul 26 16:10:15.564033 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.563999    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"]
Jul 26 16:10:15.564365 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.564329    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg" podUID="a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47" containerName="payload-processing" containerID="cri-o://c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9" gracePeriod=30
Jul 26 16:10:15.571682 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.571531    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"]
Jul 26 16:10:15.572472 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.572426    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz" podUID="d1fcea56-d53b-42da-82ee-650eb7068d1e" containerName="payload-pre-processing" containerID="cri-o://e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36" gracePeriod=30
Jul 26 16:10:15.854595 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.854569    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:10:15.856844 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.856827    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:10:15.857152 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.857134    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-plugins-config-volume\") pod \"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47\" (UID: \"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47\") "
Jul 26 16:10:15.857205 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.857171    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2zqn\" (UniqueName: \"kubernetes.io/projected/91a5b868-4220-4d33-87cd-c44662751d2f-kube-api-access-r2zqn\") pod \"91a5b868-4220-4d33-87cd-c44662751d2f\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") "
Jul 26 16:10:15.857205 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.857196    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a5b868-4220-4d33-87cd-c44662751d2f-openshift-service-ca\") pod \"91a5b868-4220-4d33-87cd-c44662751d2f\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") "
Jul 26 16:10:15.857351 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.857310    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmbpw\" (UniqueName: \"kubernetes.io/projected/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-kube-api-access-zmbpw\") pod \"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47\" (UID: \"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47\") "
Jul 26 16:10:15.857446 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.857370    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a5b868-4220-4d33-87cd-c44662751d2f-maas-api-tls\") pod \"91a5b868-4220-4d33-87cd-c44662751d2f\" (UID: \"91a5b868-4220-4d33-87cd-c44662751d2f\") "
Jul 26 16:10:15.857530 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.857506    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91a5b868-4220-4d33-87cd-c44662751d2f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "91a5b868-4220-4d33-87cd-c44662751d2f" (UID: "91a5b868-4220-4d33-87cd-c44662751d2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:10:15.857602 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.857550    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47" (UID: "a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:10:15.857693 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.857674    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:10:15.857758 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.857698    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91a5b868-4220-4d33-87cd-c44662751d2f-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:10:15.859309 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.859283    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91a5b868-4220-4d33-87cd-c44662751d2f-kube-api-access-r2zqn" (OuterVolumeSpecName: "kube-api-access-r2zqn") pod "91a5b868-4220-4d33-87cd-c44662751d2f" (UID: "91a5b868-4220-4d33-87cd-c44662751d2f"). InnerVolumeSpecName "kube-api-access-r2zqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:10:15.859633 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.859599    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91a5b868-4220-4d33-87cd-c44662751d2f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "91a5b868-4220-4d33-87cd-c44662751d2f" (UID: "91a5b868-4220-4d33-87cd-c44662751d2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:10:15.859812 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.859793    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-kube-api-access-zmbpw" (OuterVolumeSpecName: "kube-api-access-zmbpw") pod "a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47" (UID: "a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47"). InnerVolumeSpecName "kube-api-access-zmbpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:10:15.877963 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.877944    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:10:15.958095 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.958065    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rnf9\" (UniqueName: \"kubernetes.io/projected/d1fcea56-d53b-42da-82ee-650eb7068d1e-kube-api-access-9rnf9\") pod \"d1fcea56-d53b-42da-82ee-650eb7068d1e\" (UID: \"d1fcea56-d53b-42da-82ee-650eb7068d1e\") "
Jul 26 16:10:15.958258 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.958112    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1fcea56-d53b-42da-82ee-650eb7068d1e-plugins-config-volume\") pod \"d1fcea56-d53b-42da-82ee-650eb7068d1e\" (UID: \"d1fcea56-d53b-42da-82ee-650eb7068d1e\") "
Jul 26 16:10:15.958364 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.958347    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmbpw\" (UniqueName: \"kubernetes.io/projected/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47-kube-api-access-zmbpw\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:10:15.958402 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.958372    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91a5b868-4220-4d33-87cd-c44662751d2f-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:10:15.958402 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.958386    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2zqn\" (UniqueName: \"kubernetes.io/projected/91a5b868-4220-4d33-87cd-c44662751d2f-kube-api-access-r2zqn\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:10:15.958511 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.958474    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1fcea56-d53b-42da-82ee-650eb7068d1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d1fcea56-d53b-42da-82ee-650eb7068d1e" (UID: "d1fcea56-d53b-42da-82ee-650eb7068d1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:10:15.960113 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:15.960092    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1fcea56-d53b-42da-82ee-650eb7068d1e-kube-api-access-9rnf9" (OuterVolumeSpecName: "kube-api-access-9rnf9") pod "d1fcea56-d53b-42da-82ee-650eb7068d1e" (UID: "d1fcea56-d53b-42da-82ee-650eb7068d1e"). InnerVolumeSpecName "kube-api-access-9rnf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:10:16.058721 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.058652    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rnf9\" (UniqueName: \"kubernetes.io/projected/d1fcea56-d53b-42da-82ee-650eb7068d1e-kube-api-access-9rnf9\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:10:16.058721 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.058677    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1fcea56-d53b-42da-82ee-650eb7068d1e-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:10:16.228626 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.228588    2560 generic.go:356] "Generic (PLEG): container finished" podID="a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47" containerID="c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9" exitCode=0
Jul 26 16:10:16.228792 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.228629    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg" event={"ID":"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47","Type":"ContainerDied","Data":"c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9"}
Jul 26 16:10:16.228792 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.228689    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg" event={"ID":"a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47","Type":"ContainerDied","Data":"e84377955864a173668aebb82ae1f47e933b6e193735adc323b7aa5b7d9ca16b"}
Jul 26 16:10:16.228792 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.228636    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"
Jul 26 16:10:16.229000 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.228720    2560 scope.go:117] "RemoveContainer" containerID="c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9"
Jul 26 16:10:16.229969 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.229947    2560 generic.go:356] "Generic (PLEG): container finished" podID="d1fcea56-d53b-42da-82ee-650eb7068d1e" containerID="e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36" exitCode=0
Jul 26 16:10:16.230078 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.229988    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"
Jul 26 16:10:16.230078 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.230027    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz" event={"ID":"d1fcea56-d53b-42da-82ee-650eb7068d1e","Type":"ContainerDied","Data":"e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36"}
Jul 26 16:10:16.230198 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.230078    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz" event={"ID":"d1fcea56-d53b-42da-82ee-650eb7068d1e","Type":"ContainerDied","Data":"b4309316c884cf463756b44f51a9740f85551810f1db2fb993e3a0048a3a53ff"}
Jul 26 16:10:16.231488 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.231462    2560 generic.go:356] "Generic (PLEG): container finished" podID="91a5b868-4220-4d33-87cd-c44662751d2f" containerID="523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962" exitCode=0
Jul 26 16:10:16.231608 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.231497    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"
Jul 26 16:10:16.231608 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.231517    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr" event={"ID":"91a5b868-4220-4d33-87cd-c44662751d2f","Type":"ContainerDied","Data":"523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962"}
Jul 26 16:10:16.231608 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.231557    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr" event={"ID":"91a5b868-4220-4d33-87cd-c44662751d2f","Type":"ContainerDied","Data":"5c76a8c339e21e906ca1a4a47759cad6dff94eada20c042b744112455bf8e4ed"}
Jul 26 16:10:16.239006 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.238986    2560 scope.go:117] "RemoveContainer" containerID="c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9"
Jul 26 16:10:16.239293 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:10:16.239265    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9\": container with ID starting with c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9 not found: ID does not exist" containerID="c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9"
Jul 26 16:10:16.239398 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.239301    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9"} err="failed to get container status \"c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9\": rpc error: code = NotFound desc = could not find container \"c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9\": container with ID starting with c2c67daee2d0805904a4cbe5971f9fe6e1d9688c87045934df0851ea0df944b9 not found: ID does not exist"
Jul 26 16:10:16.239398 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.239324    2560 scope.go:117] "RemoveContainer" containerID="e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36"
Jul 26 16:10:16.247822 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.247805    2560 scope.go:117] "RemoveContainer" containerID="e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36"
Jul 26 16:10:16.248087 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:10:16.248069    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36\": container with ID starting with e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36 not found: ID does not exist" containerID="e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36"
Jul 26 16:10:16.248168 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.248096    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36"} err="failed to get container status \"e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36\": rpc error: code = NotFound desc = could not find container \"e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36\": container with ID starting with e422088fb3e5d28bc9d52c645fc04828608fe6e377a7a0f1ff59a063ea792f36 not found: ID does not exist"
Jul 26 16:10:16.248168 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.248119    2560 scope.go:117] "RemoveContainer" containerID="523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962"
Jul 26 16:10:16.255561 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.255469    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"]
Jul 26 16:10:16.255615 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.255593    2560 scope.go:117] "RemoveContainer" containerID="523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962"
Jul 26 16:10:16.255846 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:10:16.255831    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962\": container with ID starting with 523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962 not found: ID does not exist" containerID="523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962"
Jul 26 16:10:16.255913 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.255852    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962"} err="failed to get container status \"523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962\": rpc error: code = NotFound desc = could not find container \"523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962\": container with ID starting with 523e124602732a71bba7b9c3021e280837fdfa27ef5822728d415030e39bb962 not found: ID does not exist"
Jul 26 16:10:16.259569 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.259546    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a44779b-979f499b-d2rvz"]
Jul 26 16:10:16.271771 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.271750    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"]
Jul 26 16:10:16.275137 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.275119    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a44779b-65dfdd9f9d-p4ltg"]
Jul 26 16:10:16.285016 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.284998    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"]
Jul 26 16:10:16.288860 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:16.288839    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a44779b-79b4894999-lfxrr"]
Jul 26 16:10:17.138766 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:17.138725    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91a5b868-4220-4d33-87cd-c44662751d2f" path="/var/lib/kubelet/pods/91a5b868-4220-4d33-87cd-c44662751d2f/volumes"
Jul 26 16:10:17.139279 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:17.139255    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47" path="/var/lib/kubelet/pods/a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47/volumes"
Jul 26 16:10:17.139640 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:17.139619    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1fcea56-d53b-42da-82ee-650eb7068d1e" path="/var/lib/kubelet/pods/d1fcea56-d53b-42da-82ee-650eb7068d1e/volumes"
Jul 26 16:10:20.076547 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:20.076517    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"]
Jul 26 16:10:20.078762 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:20.078737    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a44779b-tgn6w"]
Jul 26 16:10:21.137905 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:10:21.137854    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5effbd5-2546-43ae-b605-4cb13eff02cc" path="/var/lib/kubelet/pods/d5effbd5-2546-43ae-b605-4cb13eff02cc/volumes"
Jul 26 16:11:05.160579 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:11:05.160549    2560 scope.go:117] "RemoveContainer" containerID="09f79c2d1b2dd993ee78c53c5bce60fe6a1b4d6f20c4e54e456726877327fbab"
Jul 26 16:11:05.168565 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:11:05.168534    2560 scope.go:117] "RemoveContainer" containerID="ad567771b2137c7720bbe450d52e50f2f5babb36fbd12fce780e4b87a73c4205"
Jul 26 16:12:05.198741 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:05.198713    2560 scope.go:117] "RemoveContainer" containerID="8396549ae4d8f9f6405f93ef714c26877e55b9e5b946d3c516794528b79936f7"
Jul 26 16:12:09.357282 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357249    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"]
Jul 26 16:12:09.357735 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357718    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1fcea56-d53b-42da-82ee-650eb7068d1e" containerName="payload-pre-processing"
Jul 26 16:12:09.357791 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357738    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1fcea56-d53b-42da-82ee-650eb7068d1e" containerName="payload-pre-processing"
Jul 26 16:12:09.357791 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357755    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91a5b868-4220-4d33-87cd-c44662751d2f" containerName="maas-api"
Jul 26 16:12:09.357791 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357764    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="91a5b868-4220-4d33-87cd-c44662751d2f" containerName="maas-api"
Jul 26 16:12:09.357791 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357785    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5effbd5-2546-43ae-b605-4cb13eff02cc" containerName="revoke-keys"
Jul 26 16:12:09.357967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357795    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5effbd5-2546-43ae-b605-4cb13eff02cc" containerName="revoke-keys"
Jul 26 16:12:09.357967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357808    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47" containerName="payload-processing"
Jul 26 16:12:09.357967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357817    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47" containerName="payload-processing"
Jul 26 16:12:09.357967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357929    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5effbd5-2546-43ae-b605-4cb13eff02cc" containerName="revoke-keys"
Jul 26 16:12:09.357967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357942    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="91a5b868-4220-4d33-87cd-c44662751d2f" containerName="maas-api"
Jul 26 16:12:09.357967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357951    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1fcea56-d53b-42da-82ee-650eb7068d1e" containerName="payload-pre-processing"
Jul 26 16:12:09.357967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.357961    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3e58e4c-04ba-41a9-9af3-ecc46bf8ee47" containerName="payload-processing"
Jul 26 16:12:09.361203 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.361183    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:09.363349 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.363322    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7c7176-serving-cert\""
Jul 26 16:12:09.371513 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.371491    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"]
Jul 26 16:12:09.440981 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.440958    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctlsl\" (UniqueName: \"kubernetes.io/projected/912bcb85-8528-41d3-8e40-75eab3124425-kube-api-access-ctlsl\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:09.441117 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.441031    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/912bcb85-8528-41d3-8e40-75eab3124425-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:09.441117 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.441075    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:09.542266 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.542242    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctlsl\" (UniqueName: \"kubernetes.io/projected/912bcb85-8528-41d3-8e40-75eab3124425-kube-api-access-ctlsl\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:09.542377 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.542294    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/912bcb85-8528-41d3-8e40-75eab3124425-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:09.542377 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.542323    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:09.542495 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:12:09.542431    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-serving-cert: secret "maas-api-e2e-inf-a-7c7176-serving-cert" not found
Jul 26 16:12:09.542546 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:12:09.542503    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls podName:912bcb85-8528-41d3-8e40-75eab3124425 nodeName:}" failed. No retries permitted until 2026-07-26 16:12:10.042488555 +0000 UTC m=+2045.541878371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls") pod "maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64" (UID: "912bcb85-8528-41d3-8e40-75eab3124425") : secret "maas-api-e2e-inf-a-7c7176-serving-cert" not found
Jul 26 16:12:09.543035 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.543014    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/912bcb85-8528-41d3-8e40-75eab3124425-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:09.559350 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:09.559330    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctlsl\" (UniqueName: \"kubernetes.io/projected/912bcb85-8528-41d3-8e40-75eab3124425-kube-api-access-ctlsl\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:10.046283 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.046250    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:10.046473 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:12:10.046395    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-serving-cert: secret "maas-api-e2e-inf-a-7c7176-serving-cert" not found
Jul 26 16:12:10.046473 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:12:10.046451    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls podName:912bcb85-8528-41d3-8e40-75eab3124425 nodeName:}" failed. No retries permitted until 2026-07-26 16:12:11.046438513 +0000 UTC m=+2046.545828329 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls") pod "maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64" (UID: "912bcb85-8528-41d3-8e40-75eab3124425") : secret "maas-api-e2e-inf-a-7c7176-serving-cert" not found
Jul 26 16:12:10.137656 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.137623    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"]
Jul 26 16:12:10.141016 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.140997    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:12:10.143288 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.143266    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-7c7176-dockercfg-zhgrq\""
Jul 26 16:12:10.143390 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.143301    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-7c7176\""
Jul 26 16:12:10.152397 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.152367    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"]
Jul 26 16:12:10.174529 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.174509    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"]
Jul 26 16:12:10.177748 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.177732    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:12:10.188603 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.188581    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"]
Jul 26 16:12:10.247177 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.247154    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb2853b4-2b86-458f-96e6-dac50dd61f1a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv\" (UID: \"cb2853b4-2b86-458f-96e6-dac50dd61f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:12:10.247293 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.247210    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrwpf\" (UniqueName: \"kubernetes.io/projected/cb2853b4-2b86-458f-96e6-dac50dd61f1a-kube-api-access-lrwpf\") pod \"payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv\" (UID: \"cb2853b4-2b86-458f-96e6-dac50dd61f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:12:10.347943 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.347869    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz9v9\" (UniqueName: \"kubernetes.io/projected/b7dfd566-180c-4230-8f06-6509d7898006-kube-api-access-hz9v9\") pod \"payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq\" (UID: \"b7dfd566-180c-4230-8f06-6509d7898006\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:12:10.347943 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.347937    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb2853b4-2b86-458f-96e6-dac50dd61f1a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv\" (UID: \"cb2853b4-2b86-458f-96e6-dac50dd61f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:12:10.348276 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.347980    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrwpf\" (UniqueName: \"kubernetes.io/projected/cb2853b4-2b86-458f-96e6-dac50dd61f1a-kube-api-access-lrwpf\") pod \"payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv\" (UID: \"cb2853b4-2b86-458f-96e6-dac50dd61f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:12:10.348276 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.348052    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7dfd566-180c-4230-8f06-6509d7898006-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq\" (UID: \"b7dfd566-180c-4230-8f06-6509d7898006\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:12:10.348533 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.348514    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb2853b4-2b86-458f-96e6-dac50dd61f1a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv\" (UID: \"cb2853b4-2b86-458f-96e6-dac50dd61f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:12:10.355522 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.355502    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrwpf\" (UniqueName: \"kubernetes.io/projected/cb2853b4-2b86-458f-96e6-dac50dd61f1a-kube-api-access-lrwpf\") pod \"payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv\" (UID: \"cb2853b4-2b86-458f-96e6-dac50dd61f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:12:10.448850 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.448820    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz9v9\" (UniqueName: \"kubernetes.io/projected/b7dfd566-180c-4230-8f06-6509d7898006-kube-api-access-hz9v9\") pod \"payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq\" (UID: \"b7dfd566-180c-4230-8f06-6509d7898006\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:12:10.449215 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.448898    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7dfd566-180c-4230-8f06-6509d7898006-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq\" (UID: \"b7dfd566-180c-4230-8f06-6509d7898006\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:12:10.449502 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.449484    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7dfd566-180c-4230-8f06-6509d7898006-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq\" (UID: \"b7dfd566-180c-4230-8f06-6509d7898006\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:12:10.451764 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.451747    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:12:10.456479 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.456458    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz9v9\" (UniqueName: \"kubernetes.io/projected/b7dfd566-180c-4230-8f06-6509d7898006-kube-api-access-hz9v9\") pod \"payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq\" (UID: \"b7dfd566-180c-4230-8f06-6509d7898006\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:12:10.487473 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.487446    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:12:10.797569 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.797543    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"]
Jul 26 16:12:10.800134 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:12:10.800099    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb2853b4_2b86_458f_96e6_dac50dd61f1a.slice/crio-1d5dbe84d3e994bfaab0a8546e169e6937189875a750f4eeb5c1b363b912f4a8 WatchSource:0}: Error finding container 1d5dbe84d3e994bfaab0a8546e169e6937189875a750f4eeb5c1b363b912f4a8: Status 404 returned error can't find the container with id 1d5dbe84d3e994bfaab0a8546e169e6937189875a750f4eeb5c1b363b912f4a8
Jul 26 16:12:10.832140 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:10.832118    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"]
Jul 26 16:12:10.840035 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:12:10.840008    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7dfd566_180c_4230_8f06_6509d7898006.slice/crio-3f008414c5c1ab4e6de62e58795e6d59daea49d317d93e5ea962f67a466d0e20 WatchSource:0}: Error finding container 3f008414c5c1ab4e6de62e58795e6d59daea49d317d93e5ea962f67a466d0e20: Status 404 returned error can't find the container with id 3f008414c5c1ab4e6de62e58795e6d59daea49d317d93e5ea962f67a466d0e20
Jul 26 16:12:11.054990 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.054911    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:11.057188 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.057159    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls\") pod \"maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:11.173226 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.173199    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:11.315210 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.314929    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"]
Jul 26 16:12:11.315611 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:12:11.315582    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod912bcb85_8528_41d3_8e40_75eab3124425.slice/crio-d31a10425d0ec6a85f1c892b686746c9ccab31c34d84e93312686a75e957e49a WatchSource:0}: Error finding container d31a10425d0ec6a85f1c892b686746c9ccab31c34d84e93312686a75e957e49a: Status 404 returned error can't find the container with id d31a10425d0ec6a85f1c892b686746c9ccab31c34d84e93312686a75e957e49a
Jul 26 16:12:11.738581 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.738550    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq" event={"ID":"b7dfd566-180c-4230-8f06-6509d7898006","Type":"ContainerStarted","Data":"50e8e60722c5ae90cd80e876c293d8b71b5c096eeca8ed1e415044dcfddab9d9"}
Jul 26 16:12:11.738967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.738616    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq" event={"ID":"b7dfd566-180c-4230-8f06-6509d7898006","Type":"ContainerStarted","Data":"3f008414c5c1ab4e6de62e58795e6d59daea49d317d93e5ea962f67a466d0e20"}
Jul 26 16:12:11.738967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.738651    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:12:11.739749 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.739725    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64" event={"ID":"912bcb85-8528-41d3-8e40-75eab3124425","Type":"ContainerStarted","Data":"d31a10425d0ec6a85f1c892b686746c9ccab31c34d84e93312686a75e957e49a"}
Jul 26 16:12:11.741217 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.741192    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv" event={"ID":"cb2853b4-2b86-458f-96e6-dac50dd61f1a","Type":"ContainerStarted","Data":"704cb9eafa24e1efbaa596f15f71b07dea794ff3086e23810cb2266ba5f8cc3b"}
Jul 26 16:12:11.741316 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.741242    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv" event={"ID":"cb2853b4-2b86-458f-96e6-dac50dd61f1a","Type":"ContainerStarted","Data":"1d5dbe84d3e994bfaab0a8546e169e6937189875a750f4eeb5c1b363b912f4a8"}
Jul 26 16:12:11.741316 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.741272    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:12:11.756391 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.756345    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq" podStartSLOduration=1.7563322019999998 podStartE2EDuration="1.756332202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:12:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:12:11.755242004 +0000 UTC m=+2047.254631841" watchObservedRunningTime="2026-07-26 16:12:11.756332202 +0000 UTC m=+2047.255722018"
Jul 26 16:12:11.771137 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:11.771095    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv" podStartSLOduration=1.771081006 podStartE2EDuration="1.771081006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:12:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:12:11.769084156 +0000 UTC m=+2047.268473995" watchObservedRunningTime="2026-07-26 16:12:11.771081006 +0000 UTC m=+2047.270470843"
Jul 26 16:12:12.748068 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:12.748026    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64" event={"ID":"912bcb85-8528-41d3-8e40-75eab3124425","Type":"ContainerStarted","Data":"fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639"}
Jul 26 16:12:12.748522 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:12.748316    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:12.764863 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:12.764816    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64" podStartSLOduration=3.389153501 podStartE2EDuration="3.764802073s" totalImagesPullingTime="375.648572ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:12:12.762670315 +0000 UTC m=+2048.262060153" watchObservedRunningTime="2026-07-26 16:12:12.764802073 +0000 UTC m=+2048.264191911"
Jul 26 16:12:17.332751 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.332715    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"]
Jul 26 16:12:17.336697 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.336673    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.339802 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.339777    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-959cca-serving-cert\""
Jul 26 16:12:17.363705 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.363683    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"]
Jul 26 16:12:17.504835 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.504804    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5450c6b0-b78f-45f4-afc8-1bb34177e85e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.504835 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.504837    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wshhm\" (UniqueName: \"kubernetes.io/projected/5450c6b0-b78f-45f4-afc8-1bb34177e85e-kube-api-access-wshhm\") pod \"maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.505047 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.504924    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5450c6b0-b78f-45f4-afc8-1bb34177e85e-maas-api-tls\") pod \"maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.606168 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.606102    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5450c6b0-b78f-45f4-afc8-1bb34177e85e-maas-api-tls\") pod \"maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.606168 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.606164    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5450c6b0-b78f-45f4-afc8-1bb34177e85e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.606327 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.606184    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wshhm\" (UniqueName: \"kubernetes.io/projected/5450c6b0-b78f-45f4-afc8-1bb34177e85e-kube-api-access-wshhm\") pod \"maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.606838 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.606810    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5450c6b0-b78f-45f4-afc8-1bb34177e85e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.608630 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.608603    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5450c6b0-b78f-45f4-afc8-1bb34177e85e-maas-api-tls\") pod \"maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.614569 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.614546    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wshhm\" (UniqueName: \"kubernetes.io/projected/5450c6b0-b78f-45f4-afc8-1bb34177e85e-kube-api-access-wshhm\") pod \"maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.648483 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.648459    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:17.783594 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.783535    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"]
Jul 26 16:12:17.786296 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:12:17.786270    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5450c6b0_b78f_45f4_afc8_1bb34177e85e.slice/crio-de79107869cf3c2214f118554e99581056cd0579c00a075a6bd14dc0dc4900f2 WatchSource:0}: Error finding container de79107869cf3c2214f118554e99581056cd0579c00a075a6bd14dc0dc4900f2: Status 404 returned error can't find the container with id de79107869cf3c2214f118554e99581056cd0579c00a075a6bd14dc0dc4900f2
Jul 26 16:12:17.988701 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.988663    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"]
Jul 26 16:12:17.993200 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.993177    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:12:17.996632 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.996605    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-959cca\""
Jul 26 16:12:17.996784 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:17.996767    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-959cca-dockercfg-z828s\""
Jul 26 16:12:18.010576 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.010552    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"]
Jul 26 16:12:18.025361 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.025339    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"]
Jul 26 16:12:18.028560 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.028545    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:12:18.039471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.039452    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"]
Jul 26 16:12:18.111046 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.111016    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-828ks\" (UniqueName: \"kubernetes.io/projected/651b5af6-9f4f-415f-b2de-f65ec79e0887-kube-api-access-828ks\") pod \"payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp\" (UID: \"651b5af6-9f4f-415f-b2de-f65ec79e0887\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:12:18.111172 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.111155    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/651b5af6-9f4f-415f-b2de-f65ec79e0887-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp\" (UID: \"651b5af6-9f4f-415f-b2de-f65ec79e0887\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:12:18.212145 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.212110    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-828ks\" (UniqueName: \"kubernetes.io/projected/651b5af6-9f4f-415f-b2de-f65ec79e0887-kube-api-access-828ks\") pod \"payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp\" (UID: \"651b5af6-9f4f-415f-b2de-f65ec79e0887\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:12:18.212283 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.212152    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqzsn\" (UniqueName: \"kubernetes.io/projected/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-kube-api-access-dqzsn\") pod \"payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v\" (UID: \"8160d202-0a6d-4aa2-b98b-f2e8d7b17694\") " pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:12:18.212283 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.212195    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v\" (UID: \"8160d202-0a6d-4aa2-b98b-f2e8d7b17694\") " pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:12:18.212405 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.212331    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/651b5af6-9f4f-415f-b2de-f65ec79e0887-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp\" (UID: \"651b5af6-9f4f-415f-b2de-f65ec79e0887\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:12:18.212980 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.212960    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/651b5af6-9f4f-415f-b2de-f65ec79e0887-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp\" (UID: \"651b5af6-9f4f-415f-b2de-f65ec79e0887\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:12:18.219972 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.219951    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-828ks\" (UniqueName: \"kubernetes.io/projected/651b5af6-9f4f-415f-b2de-f65ec79e0887-kube-api-access-828ks\") pod \"payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp\" (UID: \"651b5af6-9f4f-415f-b2de-f65ec79e0887\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:12:18.304011 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.303958    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:12:18.313251 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.313225    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqzsn\" (UniqueName: \"kubernetes.io/projected/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-kube-api-access-dqzsn\") pod \"payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v\" (UID: \"8160d202-0a6d-4aa2-b98b-f2e8d7b17694\") " pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:12:18.313358 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.313299    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v\" (UID: \"8160d202-0a6d-4aa2-b98b-f2e8d7b17694\") " pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:12:18.314129 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.314104    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v\" (UID: \"8160d202-0a6d-4aa2-b98b-f2e8d7b17694\") " pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:12:18.320444 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.320422    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqzsn\" (UniqueName: \"kubernetes.io/projected/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-kube-api-access-dqzsn\") pod \"payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v\" (UID: \"8160d202-0a6d-4aa2-b98b-f2e8d7b17694\") " pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:12:18.338022 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.338002    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:12:18.446991 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.446963    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"]
Jul 26 16:12:18.449142 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:12:18.449113    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod651b5af6_9f4f_415f_b2de_f65ec79e0887.slice/crio-2d1e5d19904c643d70ab005c87cb82d26a8b7e08b2e05adcfdf1e6b329bba608 WatchSource:0}: Error finding container 2d1e5d19904c643d70ab005c87cb82d26a8b7e08b2e05adcfdf1e6b329bba608: Status 404 returned error can't find the container with id 2d1e5d19904c643d70ab005c87cb82d26a8b7e08b2e05adcfdf1e6b329bba608
Jul 26 16:12:18.694726 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.694388    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"]
Jul 26 16:12:18.697286 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:12:18.697259    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8160d202_0a6d_4aa2_b98b_f2e8d7b17694.slice/crio-26c285e031c137c7dfbdd694ea0a1425d5e8fe35728959ac130e8d412e476475 WatchSource:0}: Error finding container 26c285e031c137c7dfbdd694ea0a1425d5e8fe35728959ac130e8d412e476475: Status 404 returned error can't find the container with id 26c285e031c137c7dfbdd694ea0a1425d5e8fe35728959ac130e8d412e476475
Jul 26 16:12:18.764260 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.764239    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:12:18.793392 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.793214    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v" event={"ID":"8160d202-0a6d-4aa2-b98b-f2e8d7b17694","Type":"ContainerStarted","Data":"61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4"}
Jul 26 16:12:18.793392 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.793329    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v" event={"ID":"8160d202-0a6d-4aa2-b98b-f2e8d7b17694","Type":"ContainerStarted","Data":"26c285e031c137c7dfbdd694ea0a1425d5e8fe35728959ac130e8d412e476475"}
Jul 26 16:12:18.793590 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.793406    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:12:18.796453 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.795732    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg" event={"ID":"5450c6b0-b78f-45f4-afc8-1bb34177e85e","Type":"ContainerStarted","Data":"e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131"}
Jul 26 16:12:18.796453 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.795779    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg" event={"ID":"5450c6b0-b78f-45f4-afc8-1bb34177e85e","Type":"ContainerStarted","Data":"de79107869cf3c2214f118554e99581056cd0579c00a075a6bd14dc0dc4900f2"}
Jul 26 16:12:18.796453 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.795806    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:18.797980 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.797955    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp" event={"ID":"651b5af6-9f4f-415f-b2de-f65ec79e0887","Type":"ContainerStarted","Data":"0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5"}
Jul 26 16:12:18.798106 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.797999    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp" event={"ID":"651b5af6-9f4f-415f-b2de-f65ec79e0887","Type":"ContainerStarted","Data":"2d1e5d19904c643d70ab005c87cb82d26a8b7e08b2e05adcfdf1e6b329bba608"}
Jul 26 16:12:18.798106 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.798040    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:12:18.809332 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.809294    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v" podStartSLOduration=0.809279435 podStartE2EDuration="809.279435ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:12:18.807792469 +0000 UTC m=+2054.307182323" watchObservedRunningTime="2026-07-26 16:12:18.809279435 +0000 UTC m=+2054.308669274"
Jul 26 16:12:18.825382 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.825335    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg" podStartSLOduration=1.5422376500000001 podStartE2EDuration="1.82531737s" totalImagesPullingTime="283.07972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:12:18.822701305 +0000 UTC m=+2054.322091144" watchObservedRunningTime="2026-07-26 16:12:18.82531737 +0000 UTC m=+2054.324707209"
Jul 26 16:12:18.837003 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:18.836962    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp" podStartSLOduration=1.836949728 podStartE2EDuration="1.836949728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:12:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:12:18.835592596 +0000 UTC m=+2054.334982436" watchObservedRunningTime="2026-07-26 16:12:18.836949728 +0000 UTC m=+2054.336339565"
Jul 26 16:12:22.750455 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:22.750422    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:12:22.750826 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:22.750487    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:12:24.808478 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:24.808448    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:12:29.804610 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:29.804527    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:12:29.805156 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:12:29.804710    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:13:05.215455 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:05.215419    2560 scope.go:117] "RemoveContainer" containerID="ed160cca90b8648551a91716b3cf9e3ead0b6a6ddd8f74bafe7c90fd1c0a7dad"
Jul 26 16:13:52.631145 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:52.631111    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"]
Jul 26 16:13:52.634469 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:52.634452    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"
Jul 26 16:13:52.636618 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:52.636599    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:13:52.643542 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:52.643498    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"]
Jul 26 16:13:52.803818 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:52.803792    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/108b41bf-d628-410e-9039-3dee26deada2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54\" (UID: \"108b41bf-d628-410e-9039-3dee26deada2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"
Jul 26 16:13:52.904310 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:52.904284    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/108b41bf-d628-410e-9039-3dee26deada2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54\" (UID: \"108b41bf-d628-410e-9039-3dee26deada2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"
Jul 26 16:13:52.904859 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:52.904841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/108b41bf-d628-410e-9039-3dee26deada2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54\" (UID: \"108b41bf-d628-410e-9039-3dee26deada2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"
Jul 26 16:13:52.951361 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:52.951341    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"
Jul 26 16:13:53.081156 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:53.081134    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"]
Jul 26 16:13:53.083328 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:13:53.083301    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108b41bf_d628_410e_9039_3dee26deada2.slice/crio-f770aac1e9dd25a31f2ada290e335489baba57a118146b6c27038869c3c47115 WatchSource:0}: Error finding container f770aac1e9dd25a31f2ada290e335489baba57a118146b6c27038869c3c47115: Status 404 returned error can't find the container with id f770aac1e9dd25a31f2ada290e335489baba57a118146b6c27038869c3c47115
Jul 26 16:13:53.234799 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:53.234739    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54" event={"ID":"108b41bf-d628-410e-9039-3dee26deada2","Type":"ContainerStarted","Data":"f770aac1e9dd25a31f2ada290e335489baba57a118146b6c27038869c3c47115"}
Jul 26 16:13:54.241169 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:54.241133    2560 generic.go:356] "Generic (PLEG): container finished" podID="108b41bf-d628-410e-9039-3dee26deada2" containerID="a8b8b69da70c0b453e146af12674042cecad08ed305705f85879a11091237507" exitCode=0
Jul 26 16:13:54.241579 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:54.241202    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54" event={"ID":"108b41bf-d628-410e-9039-3dee26deada2","Type":"ContainerDied","Data":"a8b8b69da70c0b453e146af12674042cecad08ed305705f85879a11091237507"}
Jul 26 16:13:55.396348 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:55.396320    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"
Jul 26 16:13:55.422388 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:55.422362    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/108b41bf-d628-410e-9039-3dee26deada2-maas-api-service-ca\") pod \"108b41bf-d628-410e-9039-3dee26deada2\" (UID: \"108b41bf-d628-410e-9039-3dee26deada2\") "
Jul 26 16:13:55.422750 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:55.422719    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/108b41bf-d628-410e-9039-3dee26deada2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "108b41bf-d628-410e-9039-3dee26deada2" (UID: "108b41bf-d628-410e-9039-3dee26deada2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:13:55.523647 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:55.523577    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/108b41bf-d628-410e-9039-3dee26deada2-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:13:56.251203 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:56.251177    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"
Jul 26 16:13:56.251361 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:56.251211    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54" event={"ID":"108b41bf-d628-410e-9039-3dee26deada2","Type":"ContainerDied","Data":"f770aac1e9dd25a31f2ada290e335489baba57a118146b6c27038869c3c47115"}
Jul 26 16:13:56.251361 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:13:56.251254    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f770aac1e9dd25a31f2ada290e335489baba57a118146b6c27038869c3c47115"
Jul 26 16:14:03.014310 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.014218    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"]
Jul 26 16:14:03.014755 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.014463    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64" podUID="912bcb85-8528-41d3-8e40-75eab3124425" containerName="maas-api" containerID="cri-o://fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639" gracePeriod=30
Jul 26 16:14:03.068842 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.068813    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"]
Jul 26 16:14:03.069113 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.069075    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq" podUID="b7dfd566-180c-4230-8f06-6509d7898006" containerName="payload-processing" containerID="cri-o://50e8e60722c5ae90cd80e876c293d8b71b5c096eeca8ed1e415044dcfddab9d9" gracePeriod=30
Jul 26 16:14:03.073808 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.073778    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"]
Jul 26 16:14:03.074149 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.074108    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv" podUID="cb2853b4-2b86-458f-96e6-dac50dd61f1a" containerName="payload-pre-processing" containerID="cri-o://704cb9eafa24e1efbaa596f15f71b07dea794ff3086e23810cb2266ba5f8cc3b" gracePeriod=30
Jul 26 16:14:03.287469 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.287436    2560 generic.go:356] "Generic (PLEG): container finished" podID="cb2853b4-2b86-458f-96e6-dac50dd61f1a" containerID="704cb9eafa24e1efbaa596f15f71b07dea794ff3086e23810cb2266ba5f8cc3b" exitCode=0
Jul 26 16:14:03.287616 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.287578    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv" event={"ID":"cb2853b4-2b86-458f-96e6-dac50dd61f1a","Type":"ContainerDied","Data":"704cb9eafa24e1efbaa596f15f71b07dea794ff3086e23810cb2266ba5f8cc3b"}
Jul 26 16:14:03.289667 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.289599    2560 generic.go:356] "Generic (PLEG): container finished" podID="b7dfd566-180c-4230-8f06-6509d7898006" containerID="50e8e60722c5ae90cd80e876c293d8b71b5c096eeca8ed1e415044dcfddab9d9" exitCode=0
Jul 26 16:14:03.289667 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.289640    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq" event={"ID":"b7dfd566-180c-4230-8f06-6509d7898006","Type":"ContainerDied","Data":"50e8e60722c5ae90cd80e876c293d8b71b5c096eeca8ed1e415044dcfddab9d9"}
Jul 26 16:14:03.367764 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.367742    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:14:03.370790 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.370770    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:14:03.378030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.378004    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb2853b4-2b86-458f-96e6-dac50dd61f1a-plugins-config-volume\") pod \"cb2853b4-2b86-458f-96e6-dac50dd61f1a\" (UID: \"cb2853b4-2b86-458f-96e6-dac50dd61f1a\") "
Jul 26 16:14:03.378126 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.378059    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7dfd566-180c-4230-8f06-6509d7898006-plugins-config-volume\") pod \"b7dfd566-180c-4230-8f06-6509d7898006\" (UID: \"b7dfd566-180c-4230-8f06-6509d7898006\") "
Jul 26 16:14:03.378126 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.378119    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz9v9\" (UniqueName: \"kubernetes.io/projected/b7dfd566-180c-4230-8f06-6509d7898006-kube-api-access-hz9v9\") pod \"b7dfd566-180c-4230-8f06-6509d7898006\" (UID: \"b7dfd566-180c-4230-8f06-6509d7898006\") "
Jul 26 16:14:03.378251 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.378171    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrwpf\" (UniqueName: \"kubernetes.io/projected/cb2853b4-2b86-458f-96e6-dac50dd61f1a-kube-api-access-lrwpf\") pod \"cb2853b4-2b86-458f-96e6-dac50dd61f1a\" (UID: \"cb2853b4-2b86-458f-96e6-dac50dd61f1a\") "
Jul 26 16:14:03.378424 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.378399    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb2853b4-2b86-458f-96e6-dac50dd61f1a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cb2853b4-2b86-458f-96e6-dac50dd61f1a" (UID: "cb2853b4-2b86-458f-96e6-dac50dd61f1a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:14:03.378478 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.378411    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7dfd566-180c-4230-8f06-6509d7898006-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7dfd566-180c-4230-8f06-6509d7898006" (UID: "b7dfd566-180c-4230-8f06-6509d7898006"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:14:03.380176 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.380142    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb2853b4-2b86-458f-96e6-dac50dd61f1a-kube-api-access-lrwpf" (OuterVolumeSpecName: "kube-api-access-lrwpf") pod "cb2853b4-2b86-458f-96e6-dac50dd61f1a" (UID: "cb2853b4-2b86-458f-96e6-dac50dd61f1a"). InnerVolumeSpecName "kube-api-access-lrwpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:14:03.380698 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.380671    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7dfd566-180c-4230-8f06-6509d7898006-kube-api-access-hz9v9" (OuterVolumeSpecName: "kube-api-access-hz9v9") pod "b7dfd566-180c-4230-8f06-6509d7898006" (UID: "b7dfd566-180c-4230-8f06-6509d7898006"). InnerVolumeSpecName "kube-api-access-hz9v9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:14:03.452142 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.452114    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:14:03.479139 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.479113    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctlsl\" (UniqueName: \"kubernetes.io/projected/912bcb85-8528-41d3-8e40-75eab3124425-kube-api-access-ctlsl\") pod \"912bcb85-8528-41d3-8e40-75eab3124425\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") "
Jul 26 16:14:03.479264 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.479162    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls\") pod \"912bcb85-8528-41d3-8e40-75eab3124425\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") "
Jul 26 16:14:03.479264 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.479195    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/912bcb85-8528-41d3-8e40-75eab3124425-openshift-service-ca\") pod \"912bcb85-8528-41d3-8e40-75eab3124425\" (UID: \"912bcb85-8528-41d3-8e40-75eab3124425\") "
Jul 26 16:14:03.479470 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.479453    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7dfd566-180c-4230-8f06-6509d7898006-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:03.479522 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.479477    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz9v9\" (UniqueName: \"kubernetes.io/projected/b7dfd566-180c-4230-8f06-6509d7898006-kube-api-access-hz9v9\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:03.479522 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.479495    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrwpf\" (UniqueName: \"kubernetes.io/projected/cb2853b4-2b86-458f-96e6-dac50dd61f1a-kube-api-access-lrwpf\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:03.479522 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.479511    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb2853b4-2b86-458f-96e6-dac50dd61f1a-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:03.479642 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.479622    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/912bcb85-8528-41d3-8e40-75eab3124425-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "912bcb85-8528-41d3-8e40-75eab3124425" (UID: "912bcb85-8528-41d3-8e40-75eab3124425"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:14:03.481183 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.481158    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "912bcb85-8528-41d3-8e40-75eab3124425" (UID: "912bcb85-8528-41d3-8e40-75eab3124425"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:14:03.481702 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.481676    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/912bcb85-8528-41d3-8e40-75eab3124425-kube-api-access-ctlsl" (OuterVolumeSpecName: "kube-api-access-ctlsl") pod "912bcb85-8528-41d3-8e40-75eab3124425" (UID: "912bcb85-8528-41d3-8e40-75eab3124425"). InnerVolumeSpecName "kube-api-access-ctlsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:14:03.580145 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.580089    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/912bcb85-8528-41d3-8e40-75eab3124425-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:03.580145 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.580109    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctlsl\" (UniqueName: \"kubernetes.io/projected/912bcb85-8528-41d3-8e40-75eab3124425-kube-api-access-ctlsl\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:03.580145 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:03.580121    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/912bcb85-8528-41d3-8e40-75eab3124425-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:04.295106 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.295085    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"
Jul 26 16:14:04.295510 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.295118    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq" event={"ID":"b7dfd566-180c-4230-8f06-6509d7898006","Type":"ContainerDied","Data":"3f008414c5c1ab4e6de62e58795e6d59daea49d317d93e5ea962f67a466d0e20"}
Jul 26 16:14:04.295510 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.295180    2560 scope.go:117] "RemoveContainer" containerID="50e8e60722c5ae90cd80e876c293d8b71b5c096eeca8ed1e415044dcfddab9d9"
Jul 26 16:14:04.296495 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.296446    2560 generic.go:356] "Generic (PLEG): container finished" podID="912bcb85-8528-41d3-8e40-75eab3124425" containerID="fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639" exitCode=0
Jul 26 16:14:04.296495 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.296490    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"
Jul 26 16:14:04.296623 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.296507    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64" event={"ID":"912bcb85-8528-41d3-8e40-75eab3124425","Type":"ContainerDied","Data":"fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639"}
Jul 26 16:14:04.296623 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.296551    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64" event={"ID":"912bcb85-8528-41d3-8e40-75eab3124425","Type":"ContainerDied","Data":"d31a10425d0ec6a85f1c892b686746c9ccab31c34d84e93312686a75e957e49a"}
Jul 26 16:14:04.297675 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.297655    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv" event={"ID":"cb2853b4-2b86-458f-96e6-dac50dd61f1a","Type":"ContainerDied","Data":"1d5dbe84d3e994bfaab0a8546e169e6937189875a750f4eeb5c1b363b912f4a8"}
Jul 26 16:14:04.297780 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.297689    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"
Jul 26 16:14:04.305403 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.305385    2560 scope.go:117] "RemoveContainer" containerID="fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639"
Jul 26 16:14:04.313650 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.313634    2560 scope.go:117] "RemoveContainer" containerID="fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639"
Jul 26 16:14:04.313929 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:14:04.313906    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639\": container with ID starting with fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639 not found: ID does not exist" containerID="fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639"
Jul 26 16:14:04.314022 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.313934    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639"} err="failed to get container status \"fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639\": rpc error: code = NotFound desc = could not find container \"fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639\": container with ID starting with fa9451876345ac5657f334cfcd334264edcc24ee9aaa3ae3d55150396826a639 not found: ID does not exist"
Jul 26 16:14:04.314022 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.313950    2560 scope.go:117] "RemoveContainer" containerID="704cb9eafa24e1efbaa596f15f71b07dea794ff3086e23810cb2266ba5f8cc3b"
Jul 26 16:14:04.317388 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.317363    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"]
Jul 26 16:14:04.321126 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.321107    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7c7176-b46cdc77b-tddzq"]
Jul 26 16:14:04.329703 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.329684    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"]
Jul 26 16:14:04.334991 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.334972    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7c7176-8cf946b75-6hg64"]
Jul 26 16:14:04.343644 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.343600    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"]
Jul 26 16:14:04.349086 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:04.349067    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7c7176-b4585b9cb-jwhhv"]
Jul 26 16:14:05.141369 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:05.141339    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="912bcb85-8528-41d3-8e40-75eab3124425" path="/var/lib/kubelet/pods/912bcb85-8528-41d3-8e40-75eab3124425/volumes"
Jul 26 16:14:05.141722 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:05.141707    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7dfd566-180c-4230-8f06-6509d7898006" path="/var/lib/kubelet/pods/b7dfd566-180c-4230-8f06-6509d7898006/volumes"
Jul 26 16:14:05.142067 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:05.142053    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb2853b4-2b86-458f-96e6-dac50dd61f1a" path="/var/lib/kubelet/pods/cb2853b4-2b86-458f-96e6-dac50dd61f1a/volumes"
Jul 26 16:14:07.712158 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:07.712125    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"]
Jul 26 16:14:07.715315 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:07.715293    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7c7176-p4d54"]
Jul 26 16:14:09.138714 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:09.138683    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="108b41bf-d628-410e-9039-3dee26deada2" path="/var/lib/kubelet/pods/108b41bf-d628-410e-9039-3dee26deada2/volumes"
Jul 26 16:14:14.297181 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297146    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"]
Jul 26 16:14:14.297669 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297647    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="912bcb85-8528-41d3-8e40-75eab3124425" containerName="maas-api"
Jul 26 16:14:14.297753 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297672    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="912bcb85-8528-41d3-8e40-75eab3124425" containerName="maas-api"
Jul 26 16:14:14.297753 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297691    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb2853b4-2b86-458f-96e6-dac50dd61f1a" containerName="payload-pre-processing"
Jul 26 16:14:14.297753 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297700    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb2853b4-2b86-458f-96e6-dac50dd61f1a" containerName="payload-pre-processing"
Jul 26 16:14:14.297753 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297729    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="108b41bf-d628-410e-9039-3dee26deada2" containerName="revoke-keys"
Jul 26 16:14:14.297753 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297738    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="108b41bf-d628-410e-9039-3dee26deada2" containerName="revoke-keys"
Jul 26 16:14:14.298017 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297754    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7dfd566-180c-4230-8f06-6509d7898006" containerName="payload-processing"
Jul 26 16:14:14.298017 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297763    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7dfd566-180c-4230-8f06-6509d7898006" containerName="payload-processing"
Jul 26 16:14:14.298017 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297848    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="912bcb85-8528-41d3-8e40-75eab3124425" containerName="maas-api"
Jul 26 16:14:14.298017 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297861    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb2853b4-2b86-458f-96e6-dac50dd61f1a" containerName="payload-pre-processing"
Jul 26 16:14:14.298017 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297873    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7dfd566-180c-4230-8f06-6509d7898006" containerName="payload-processing"
Jul 26 16:14:14.298017 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.297883    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="108b41bf-d628-410e-9039-3dee26deada2" containerName="revoke-keys"
Jul 26 16:14:14.300806 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.300785    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"
Jul 26 16:14:14.303196 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.303177    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:14:14.308109 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.308088    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"]
Jul 26 16:14:14.356115 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.356085    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9656afe8-156f-45c7-bef3-5b7d9a30efd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-959cca-cngh8\" (UID: \"9656afe8-156f-45c7-bef3-5b7d9a30efd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"
Jul 26 16:14:14.456457 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.456426    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9656afe8-156f-45c7-bef3-5b7d9a30efd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-959cca-cngh8\" (UID: \"9656afe8-156f-45c7-bef3-5b7d9a30efd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"
Jul 26 16:14:14.457066 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.457045    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9656afe8-156f-45c7-bef3-5b7d9a30efd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-959cca-cngh8\" (UID: \"9656afe8-156f-45c7-bef3-5b7d9a30efd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"
Jul 26 16:14:14.612339 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.612282    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"
Jul 26 16:14:14.739099 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:14.739070    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"]
Jul 26 16:14:14.742023 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:14:14.741993    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9656afe8_156f_45c7_bef3_5b7d9a30efd0.slice/crio-56e1bc85c799029624b034629de3d7b3ab6b1f7a22038ddeacff62708231bf60 WatchSource:0}: Error finding container 56e1bc85c799029624b034629de3d7b3ab6b1f7a22038ddeacff62708231bf60: Status 404 returned error can't find the container with id 56e1bc85c799029624b034629de3d7b3ab6b1f7a22038ddeacff62708231bf60
Jul 26 16:14:15.347524 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:15.347441    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8" event={"ID":"9656afe8-156f-45c7-bef3-5b7d9a30efd0","Type":"ContainerStarted","Data":"9a9a72369deb2fb4892ab97714e7d5d6cbae8bffbf23a1ca2f730a2027bb489e"}
Jul 26 16:14:15.347524 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:15.347500    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8" event={"ID":"9656afe8-156f-45c7-bef3-5b7d9a30efd0","Type":"ContainerStarted","Data":"56e1bc85c799029624b034629de3d7b3ab6b1f7a22038ddeacff62708231bf60"}
Jul 26 16:14:15.361125 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:15.361081    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8" podStartSLOduration=1.361064614 podStartE2EDuration="1.361064614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:14:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:14:15.360552103 +0000 UTC m=+2170.859941942" watchObservedRunningTime="2026-07-26 16:14:15.361064614 +0000 UTC m=+2170.860454453"
Jul 26 16:14:16.353755 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:16.353720    2560 generic.go:356] "Generic (PLEG): container finished" podID="9656afe8-156f-45c7-bef3-5b7d9a30efd0" containerID="9a9a72369deb2fb4892ab97714e7d5d6cbae8bffbf23a1ca2f730a2027bb489e" exitCode=0
Jul 26 16:14:16.354177 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:16.353776    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8" event={"ID":"9656afe8-156f-45c7-bef3-5b7d9a30efd0","Type":"ContainerDied","Data":"9a9a72369deb2fb4892ab97714e7d5d6cbae8bffbf23a1ca2f730a2027bb489e"}
Jul 26 16:14:17.488594 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:17.488570    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"
Jul 26 16:14:17.581859 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:17.581833    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9656afe8-156f-45c7-bef3-5b7d9a30efd0-maas-api-service-ca\") pod \"9656afe8-156f-45c7-bef3-5b7d9a30efd0\" (UID: \"9656afe8-156f-45c7-bef3-5b7d9a30efd0\") "
Jul 26 16:14:17.582257 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:17.582226    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9656afe8-156f-45c7-bef3-5b7d9a30efd0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9656afe8-156f-45c7-bef3-5b7d9a30efd0" (UID: "9656afe8-156f-45c7-bef3-5b7d9a30efd0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:14:17.682649 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:17.682625    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9656afe8-156f-45c7-bef3-5b7d9a30efd0-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:18.364558 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:18.364527    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"
Jul 26 16:14:18.364558 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:18.364541    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8" event={"ID":"9656afe8-156f-45c7-bef3-5b7d9a30efd0","Type":"ContainerDied","Data":"56e1bc85c799029624b034629de3d7b3ab6b1f7a22038ddeacff62708231bf60"}
Jul 26 16:14:18.364745 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:18.364578    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56e1bc85c799029624b034629de3d7b3ab6b1f7a22038ddeacff62708231bf60"
Jul 26 16:14:24.683735 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:24.683699    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"]
Jul 26 16:14:24.684318 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:24.683910    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg" podUID="5450c6b0-b78f-45f4-afc8-1bb34177e85e" containerName="maas-api" containerID="cri-o://e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131" gracePeriod=30
Jul 26 16:14:24.788473 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:24.788440    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"]
Jul 26 16:14:24.788711 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:24.788681    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v" podUID="8160d202-0a6d-4aa2-b98b-f2e8d7b17694" containerName="payload-processing" containerID="cri-o://61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4" gracePeriod=30
Jul 26 16:14:24.797307 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:24.797275    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"]
Jul 26 16:14:24.797729 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:24.797694    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp" podUID="651b5af6-9f4f-415f-b2de-f65ec79e0887" containerName="payload-pre-processing" containerID="cri-o://0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5" gracePeriod=30
Jul 26 16:14:24.803323 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:24.803291    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg" podUID="5450c6b0-b78f-45f4-afc8-1bb34177e85e" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.61:8443/health\": dial tcp 10.133.0.61:8443: connect: connection refused"
Jul 26 16:14:25.068418 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.068394    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:14:25.071154 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.071135    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:14:25.139547 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.139514    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqzsn\" (UniqueName: \"kubernetes.io/projected/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-kube-api-access-dqzsn\") pod \"8160d202-0a6d-4aa2-b98b-f2e8d7b17694\" (UID: \"8160d202-0a6d-4aa2-b98b-f2e8d7b17694\") "
Jul 26 16:14:25.139734 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.139588    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/651b5af6-9f4f-415f-b2de-f65ec79e0887-plugins-config-volume\") pod \"651b5af6-9f4f-415f-b2de-f65ec79e0887\" (UID: \"651b5af6-9f4f-415f-b2de-f65ec79e0887\") "
Jul 26 16:14:25.139734 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.139650    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-plugins-config-volume\") pod \"8160d202-0a6d-4aa2-b98b-f2e8d7b17694\" (UID: \"8160d202-0a6d-4aa2-b98b-f2e8d7b17694\") "
Jul 26 16:14:25.139734 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.139679    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-828ks\" (UniqueName: \"kubernetes.io/projected/651b5af6-9f4f-415f-b2de-f65ec79e0887-kube-api-access-828ks\") pod \"651b5af6-9f4f-415f-b2de-f65ec79e0887\" (UID: \"651b5af6-9f4f-415f-b2de-f65ec79e0887\") "
Jul 26 16:14:25.140032 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.140004    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/651b5af6-9f4f-415f-b2de-f65ec79e0887-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "651b5af6-9f4f-415f-b2de-f65ec79e0887" (UID: "651b5af6-9f4f-415f-b2de-f65ec79e0887"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:14:25.140165 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.140128    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8160d202-0a6d-4aa2-b98b-f2e8d7b17694" (UID: "8160d202-0a6d-4aa2-b98b-f2e8d7b17694"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:14:25.141737 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.141701    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-kube-api-access-dqzsn" (OuterVolumeSpecName: "kube-api-access-dqzsn") pod "8160d202-0a6d-4aa2-b98b-f2e8d7b17694" (UID: "8160d202-0a6d-4aa2-b98b-f2e8d7b17694"). InnerVolumeSpecName "kube-api-access-dqzsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:14:25.142002 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.141975    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/651b5af6-9f4f-415f-b2de-f65ec79e0887-kube-api-access-828ks" (OuterVolumeSpecName: "kube-api-access-828ks") pod "651b5af6-9f4f-415f-b2de-f65ec79e0887" (UID: "651b5af6-9f4f-415f-b2de-f65ec79e0887"). InnerVolumeSpecName "kube-api-access-828ks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:14:25.240816 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.240762    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:25.240816 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.240783    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-828ks\" (UniqueName: \"kubernetes.io/projected/651b5af6-9f4f-415f-b2de-f65ec79e0887-kube-api-access-828ks\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:25.240816 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.240793    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqzsn\" (UniqueName: \"kubernetes.io/projected/8160d202-0a6d-4aa2-b98b-f2e8d7b17694-kube-api-access-dqzsn\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:25.240816 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.240802    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/651b5af6-9f4f-415f-b2de-f65ec79e0887-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:25.394388 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.394360    2560 generic.go:356] "Generic (PLEG): container finished" podID="651b5af6-9f4f-415f-b2de-f65ec79e0887" containerID="0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5" exitCode=0
Jul 26 16:14:25.394511 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.394407    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"
Jul 26 16:14:25.394511 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.394447    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp" event={"ID":"651b5af6-9f4f-415f-b2de-f65ec79e0887","Type":"ContainerDied","Data":"0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5"}
Jul 26 16:14:25.394629 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.394527    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp" event={"ID":"651b5af6-9f4f-415f-b2de-f65ec79e0887","Type":"ContainerDied","Data":"2d1e5d19904c643d70ab005c87cb82d26a8b7e08b2e05adcfdf1e6b329bba608"}
Jul 26 16:14:25.394629 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.394560    2560 scope.go:117] "RemoveContainer" containerID="0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5"
Jul 26 16:14:25.395717 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.395692    2560 generic.go:356] "Generic (PLEG): container finished" podID="8160d202-0a6d-4aa2-b98b-f2e8d7b17694" containerID="61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4" exitCode=0
Jul 26 16:14:25.395827 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.395720    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v" event={"ID":"8160d202-0a6d-4aa2-b98b-f2e8d7b17694","Type":"ContainerDied","Data":"61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4"}
Jul 26 16:14:25.395827 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.395761    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v" event={"ID":"8160d202-0a6d-4aa2-b98b-f2e8d7b17694","Type":"ContainerDied","Data":"26c285e031c137c7dfbdd694ea0a1425d5e8fe35728959ac130e8d412e476475"}
Jul 26 16:14:25.395827 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.395776    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"
Jul 26 16:14:25.404177 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.404164    2560 scope.go:117] "RemoveContainer" containerID="0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5"
Jul 26 16:14:25.404405 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:14:25.404388    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5\": container with ID starting with 0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5 not found: ID does not exist" containerID="0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5"
Jul 26 16:14:25.404463 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.404412    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5"} err="failed to get container status \"0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5\": rpc error: code = NotFound desc = could not find container \"0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5\": container with ID starting with 0aab4f60e077549e517b2bec86bbb370d75f2ca8179245bfc82e56af0a9cf4d5 not found: ID does not exist"
Jul 26 16:14:25.404463 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.404427    2560 scope.go:117] "RemoveContainer" containerID="61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4"
Jul 26 16:14:25.412740 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.412719    2560 scope.go:117] "RemoveContainer" containerID="61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4"
Jul 26 16:14:25.413143 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:14:25.413120    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4\": container with ID starting with 61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4 not found: ID does not exist" containerID="61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4"
Jul 26 16:14:25.413218 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.413149    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4"} err="failed to get container status \"61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4\": rpc error: code = NotFound desc = could not find container \"61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4\": container with ID starting with 61693019de0174ad3c8c85733c13cec570e5ddd88500ff4a80285d5a6dd9e9e4 not found: ID does not exist"
Jul 26 16:14:25.426280 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.426262    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"]
Jul 26 16:14:25.438528 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.438507    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-959cca-69654757c6-cqkhp"]
Jul 26 16:14:25.454331 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.454314    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"]
Jul 26 16:14:25.460388 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.460366    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-959cca-f974cdff6-9ch5v"]
Jul 26 16:14:25.927606 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:25.927584    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:14:26.047613 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.047542    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5450c6b0-b78f-45f4-afc8-1bb34177e85e-openshift-service-ca\") pod \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") "
Jul 26 16:14:26.047613 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.047600    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5450c6b0-b78f-45f4-afc8-1bb34177e85e-maas-api-tls\") pod \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") "
Jul 26 16:14:26.047763 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.047631    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wshhm\" (UniqueName: \"kubernetes.io/projected/5450c6b0-b78f-45f4-afc8-1bb34177e85e-kube-api-access-wshhm\") pod \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\" (UID: \"5450c6b0-b78f-45f4-afc8-1bb34177e85e\") "
Jul 26 16:14:26.047865 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.047843    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5450c6b0-b78f-45f4-afc8-1bb34177e85e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5450c6b0-b78f-45f4-afc8-1bb34177e85e" (UID: "5450c6b0-b78f-45f4-afc8-1bb34177e85e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:14:26.049436 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.049406    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5450c6b0-b78f-45f4-afc8-1bb34177e85e-kube-api-access-wshhm" (OuterVolumeSpecName: "kube-api-access-wshhm") pod "5450c6b0-b78f-45f4-afc8-1bb34177e85e" (UID: "5450c6b0-b78f-45f4-afc8-1bb34177e85e"). InnerVolumeSpecName "kube-api-access-wshhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:14:26.049545 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.049483    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5450c6b0-b78f-45f4-afc8-1bb34177e85e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5450c6b0-b78f-45f4-afc8-1bb34177e85e" (UID: "5450c6b0-b78f-45f4-afc8-1bb34177e85e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:14:26.149031 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.149001    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wshhm\" (UniqueName: \"kubernetes.io/projected/5450c6b0-b78f-45f4-afc8-1bb34177e85e-kube-api-access-wshhm\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:26.149031 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.149028    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5450c6b0-b78f-45f4-afc8-1bb34177e85e-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:26.149031 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.149038    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5450c6b0-b78f-45f4-afc8-1bb34177e85e-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:14:26.400982 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.400957    2560 generic.go:356] "Generic (PLEG): container finished" podID="5450c6b0-b78f-45f4-afc8-1bb34177e85e" containerID="e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131" exitCode=0
Jul 26 16:14:26.401094 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.401002    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"
Jul 26 16:14:26.401094 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.401051    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg" event={"ID":"5450c6b0-b78f-45f4-afc8-1bb34177e85e","Type":"ContainerDied","Data":"e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131"}
Jul 26 16:14:26.401210 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.401106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg" event={"ID":"5450c6b0-b78f-45f4-afc8-1bb34177e85e","Type":"ContainerDied","Data":"de79107869cf3c2214f118554e99581056cd0579c00a075a6bd14dc0dc4900f2"}
Jul 26 16:14:26.401210 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.401138    2560 scope.go:117] "RemoveContainer" containerID="e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131"
Jul 26 16:14:26.410005 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.409985    2560 scope.go:117] "RemoveContainer" containerID="e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131"
Jul 26 16:14:26.410254 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:14:26.410237    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131\": container with ID starting with e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131 not found: ID does not exist" containerID="e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131"
Jul 26 16:14:26.410318 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.410266    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131"} err="failed to get container status \"e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131\": rpc error: code = NotFound desc = could not find container \"e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131\": container with ID starting with e6568fc4cd5973b5a49673e4697710029e2aa7b895591ee5ee44aa6266481131 not found: ID does not exist"
Jul 26 16:14:26.423436 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.423412    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"]
Jul 26 16:14:26.427093 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:26.427074    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-959cca-65d7cc4fb5-9v9zg"]
Jul 26 16:14:27.137961 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:27.137929    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5450c6b0-b78f-45f4-afc8-1bb34177e85e" path="/var/lib/kubelet/pods/5450c6b0-b78f-45f4-afc8-1bb34177e85e/volumes"
Jul 26 16:14:27.138308 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:27.138248    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="651b5af6-9f4f-415f-b2de-f65ec79e0887" path="/var/lib/kubelet/pods/651b5af6-9f4f-415f-b2de-f65ec79e0887/volumes"
Jul 26 16:14:27.138541 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:27.138528    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8160d202-0a6d-4aa2-b98b-f2e8d7b17694" path="/var/lib/kubelet/pods/8160d202-0a6d-4aa2-b98b-f2e8d7b17694/volumes"
Jul 26 16:14:29.366140 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:29.366110    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"]
Jul 26 16:14:29.368386 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:29.368362    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-959cca-cngh8"]
Jul 26 16:14:31.138167 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:31.138132    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9656afe8-156f-45c7-bef3-5b7d9a30efd0" path="/var/lib/kubelet/pods/9656afe8-156f-45c7-bef3-5b7d9a30efd0/volumes"
Jul 26 16:14:40.183641 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.183611    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"]
Jul 26 16:14:40.184038 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.183957    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9656afe8-156f-45c7-bef3-5b7d9a30efd0" containerName="revoke-keys"
Jul 26 16:14:40.184038 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.183970    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9656afe8-156f-45c7-bef3-5b7d9a30efd0" containerName="revoke-keys"
Jul 26 16:14:40.184038 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.183980    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8160d202-0a6d-4aa2-b98b-f2e8d7b17694" containerName="payload-processing"
Jul 26 16:14:40.184038 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.183989    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8160d202-0a6d-4aa2-b98b-f2e8d7b17694" containerName="payload-processing"
Jul 26 16:14:40.184038 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.184004    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="651b5af6-9f4f-415f-b2de-f65ec79e0887" containerName="payload-pre-processing"
Jul 26 16:14:40.184038 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.184009    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="651b5af6-9f4f-415f-b2de-f65ec79e0887" containerName="payload-pre-processing"
Jul 26 16:14:40.184038 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.184021    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5450c6b0-b78f-45f4-afc8-1bb34177e85e" containerName="maas-api"
Jul 26 16:14:40.184038 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.184026    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5450c6b0-b78f-45f4-afc8-1bb34177e85e" containerName="maas-api"
Jul 26 16:14:40.184299 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.184085    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9656afe8-156f-45c7-bef3-5b7d9a30efd0" containerName="revoke-keys"
Jul 26 16:14:40.184299 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.184094    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5450c6b0-b78f-45f4-afc8-1bb34177e85e" containerName="maas-api"
Jul 26 16:14:40.184299 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.184100    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="651b5af6-9f4f-415f-b2de-f65ec79e0887" containerName="payload-pre-processing"
Jul 26 16:14:40.184299 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.184105    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8160d202-0a6d-4aa2-b98b-f2e8d7b17694" containerName="payload-processing"
Jul 26 16:14:40.188710 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.188689    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.191349 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.191328    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4b0d96-serving-cert\""
Jul 26 16:14:40.197214 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.196709    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"]
Jul 26 16:14:40.252501 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.252472    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-maas-api-tls\") pod \"maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.252629 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.252523    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98jh7\" (UniqueName: \"kubernetes.io/projected/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-kube-api-access-98jh7\") pod \"maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.252629 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.252585    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-openshift-service-ca\") pod \"maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.353053 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.353032    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-maas-api-tls\") pod \"maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.353163 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.353061    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98jh7\" (UniqueName: \"kubernetes.io/projected/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-kube-api-access-98jh7\") pod \"maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.353163 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.353085    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-openshift-service-ca\") pod \"maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.353707 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.353688    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-openshift-service-ca\") pod \"maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.355708 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.355679    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-maas-api-tls\") pod \"maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.361497 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.361467    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98jh7\" (UniqueName: \"kubernetes.io/projected/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-kube-api-access-98jh7\") pod \"maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.502284 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.502221    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:40.626637 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.626590    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"]
Jul 26 16:14:40.628962 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:14:40.628933    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff62ae5_ee2c_451d_ab61_88a15cb6f388.slice/crio-2ad38849dadf6a9c79079eaf2f71768d54c25673c6eed2cd27ba9878b0784d34 WatchSource:0}: Error finding container 2ad38849dadf6a9c79079eaf2f71768d54c25673c6eed2cd27ba9878b0784d34: Status 404 returned error can't find the container with id 2ad38849dadf6a9c79079eaf2f71768d54c25673c6eed2cd27ba9878b0784d34
Jul 26 16:14:40.630360 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.630336    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:14:40.887075 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.887007    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"]
Jul 26 16:14:40.891139 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.891123    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:14:40.893327 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.893308    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4b0d96\""
Jul 26 16:14:40.893454 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.893338    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4b0d96-dockercfg-qvtvv\""
Jul 26 16:14:40.901622 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.901588    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"]
Jul 26 16:14:40.912340 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.912308    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"]
Jul 26 16:14:40.915431 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.915412    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:14:40.927373 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.927351    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"]
Jul 26 16:14:40.958318 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.958296    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z6g6\" (UniqueName: \"kubernetes.io/projected/4b10f97e-76a4-4201-b123-9e296f6609d7-kube-api-access-6z6g6\") pod \"payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf\" (UID: \"4b10f97e-76a4-4201-b123-9e296f6609d7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:14:40.958428 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:40.958325    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b10f97e-76a4-4201-b123-9e296f6609d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf\" (UID: \"4b10f97e-76a4-4201-b123-9e296f6609d7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:14:41.059619 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.059586    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z6g6\" (UniqueName: \"kubernetes.io/projected/4b10f97e-76a4-4201-b123-9e296f6609d7-kube-api-access-6z6g6\") pod \"payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf\" (UID: \"4b10f97e-76a4-4201-b123-9e296f6609d7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:14:41.059619 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.059622    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b10f97e-76a4-4201-b123-9e296f6609d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf\" (UID: \"4b10f97e-76a4-4201-b123-9e296f6609d7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:14:41.059828 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.059657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt\" (UID: \"d3f99705-5e2b-45b0-a6d3-791e8787fd2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:14:41.059828 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.059674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh4wx\" (UniqueName: \"kubernetes.io/projected/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-kube-api-access-lh4wx\") pod \"payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt\" (UID: \"d3f99705-5e2b-45b0-a6d3-791e8787fd2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:14:41.060355 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.060337    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b10f97e-76a4-4201-b123-9e296f6609d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf\" (UID: \"4b10f97e-76a4-4201-b123-9e296f6609d7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:14:41.068174 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.068148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z6g6\" (UniqueName: \"kubernetes.io/projected/4b10f97e-76a4-4201-b123-9e296f6609d7-kube-api-access-6z6g6\") pod \"payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf\" (UID: \"4b10f97e-76a4-4201-b123-9e296f6609d7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:14:41.160846 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.160817    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt\" (UID: \"d3f99705-5e2b-45b0-a6d3-791e8787fd2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:14:41.160984 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.160860    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh4wx\" (UniqueName: \"kubernetes.io/projected/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-kube-api-access-lh4wx\") pod \"payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt\" (UID: \"d3f99705-5e2b-45b0-a6d3-791e8787fd2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:14:41.161469 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.161449    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt\" (UID: \"d3f99705-5e2b-45b0-a6d3-791e8787fd2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:14:41.168001 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.167982    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh4wx\" (UniqueName: \"kubernetes.io/projected/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-kube-api-access-lh4wx\") pod \"payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt\" (UID: \"d3f99705-5e2b-45b0-a6d3-791e8787fd2e\") " pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:14:41.202183 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.202164    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:14:41.225097 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.225069    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:14:41.363830 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.363591    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"]
Jul 26 16:14:41.369447 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:14:41.369413    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b10f97e_76a4_4201_b123_9e296f6609d7.slice/crio-d57d1b6fff0ff672d1f56849e1b61710c1e3afd85e50983ef9f5ce1633571500 WatchSource:0}: Error finding container d57d1b6fff0ff672d1f56849e1b61710c1e3afd85e50983ef9f5ce1633571500: Status 404 returned error can't find the container with id d57d1b6fff0ff672d1f56849e1b61710c1e3afd85e50983ef9f5ce1633571500
Jul 26 16:14:41.468361 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.468335    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc" event={"ID":"3ff62ae5-ee2c-451d-ab61-88a15cb6f388","Type":"ContainerStarted","Data":"41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162"}
Jul 26 16:14:41.468667 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.468383    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc" event={"ID":"3ff62ae5-ee2c-451d-ab61-88a15cb6f388","Type":"ContainerStarted","Data":"2ad38849dadf6a9c79079eaf2f71768d54c25673c6eed2cd27ba9878b0784d34"}
Jul 26 16:14:41.468667 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.468408    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:41.469937 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.469911    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf" event={"ID":"4b10f97e-76a4-4201-b123-9e296f6609d7","Type":"ContainerStarted","Data":"a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510"}
Jul 26 16:14:41.470026 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.469961    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf" event={"ID":"4b10f97e-76a4-4201-b123-9e296f6609d7","Type":"ContainerStarted","Data":"d57d1b6fff0ff672d1f56849e1b61710c1e3afd85e50983ef9f5ce1633571500"}
Jul 26 16:14:41.470026 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.469983    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:14:41.484904 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.484849    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc" podStartSLOduration=0.997309466 podStartE2EDuration="1.484836126s" totalImagesPullingTime="487.52666ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:14:41.483019514 +0000 UTC m=+2196.982409351" watchObservedRunningTime="2026-07-26 16:14:41.484836126 +0000 UTC m=+2196.984225963"
Jul 26 16:14:41.499046 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.499009    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf" podStartSLOduration=1.498997415 podStartE2EDuration="1.498997415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:14:41.497379084 +0000 UTC m=+2196.996768923" watchObservedRunningTime="2026-07-26 16:14:41.498997415 +0000 UTC m=+2196.998387252"
Jul 26 16:14:41.602812 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:41.602790    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"]
Jul 26 16:14:41.605222 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:14:41.605195    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f99705_5e2b_45b0_a6d3_791e8787fd2e.slice/crio-c473cf13eb3d1530948b06d0c7225512de01ab77fa611541ad28f35d58e3175b WatchSource:0}: Error finding container c473cf13eb3d1530948b06d0c7225512de01ab77fa611541ad28f35d58e3175b: Status 404 returned error can't find the container with id c473cf13eb3d1530948b06d0c7225512de01ab77fa611541ad28f35d58e3175b
Jul 26 16:14:42.476060 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:42.476022    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt" event={"ID":"d3f99705-5e2b-45b0-a6d3-791e8787fd2e","Type":"ContainerStarted","Data":"f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b"}
Jul 26 16:14:42.476473 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:42.476080    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt" event={"ID":"d3f99705-5e2b-45b0-a6d3-791e8787fd2e","Type":"ContainerStarted","Data":"c473cf13eb3d1530948b06d0c7225512de01ab77fa611541ad28f35d58e3175b"}
Jul 26 16:14:42.476961 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:42.476925    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:14:42.492296 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:42.492249    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt" podStartSLOduration=2.4922345679999998 podStartE2EDuration="2.492234568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:14:42.491086384 +0000 UTC m=+2197.990476245" watchObservedRunningTime="2026-07-26 16:14:42.492234568 +0000 UTC m=+2197.991624405"
Jul 26 16:14:47.481598 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:47.481569    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:14:48.074953 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.074923    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"]
Jul 26 16:14:48.078267 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.078245    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.080253 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.080228    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-7cc89d-serving-cert\""
Jul 26 16:14:48.090074 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.090053    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"]
Jul 26 16:14:48.218497 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.218465    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73caccd-47ad-4840-ad29-c5ce450da310-openshift-service-ca\") pod \"maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.218668 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.218578    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn9bt\" (UniqueName: \"kubernetes.io/projected/b73caccd-47ad-4840-ad29-c5ce450da310-kube-api-access-pn9bt\") pod \"maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.218668 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.218621    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73caccd-47ad-4840-ad29-c5ce450da310-maas-api-tls\") pod \"maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.319201 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.319177    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73caccd-47ad-4840-ad29-c5ce450da310-openshift-service-ca\") pod \"maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.319327 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.319236    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn9bt\" (UniqueName: \"kubernetes.io/projected/b73caccd-47ad-4840-ad29-c5ce450da310-kube-api-access-pn9bt\") pod \"maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.319327 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.319256    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73caccd-47ad-4840-ad29-c5ce450da310-maas-api-tls\") pod \"maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.319790 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.319766    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73caccd-47ad-4840-ad29-c5ce450da310-openshift-service-ca\") pod \"maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.321652 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.321628    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73caccd-47ad-4840-ad29-c5ce450da310-maas-api-tls\") pod \"maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.327312 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.327257    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn9bt\" (UniqueName: \"kubernetes.io/projected/b73caccd-47ad-4840-ad29-c5ce450da310-kube-api-access-pn9bt\") pod \"maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.392413 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.392391    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:48.525767 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.525741    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"]
Jul 26 16:14:48.528006 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:14:48.527978    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73caccd_47ad_4840_ad29_c5ce450da310.slice/crio-207768c518fd51274160361a9d7a985b37fc66b800e4026c0541d543b0e38a21 WatchSource:0}: Error finding container 207768c518fd51274160361a9d7a985b37fc66b800e4026c0541d543b0e38a21: Status 404 returned error can't find the container with id 207768c518fd51274160361a9d7a985b37fc66b800e4026c0541d543b0e38a21
Jul 26 16:14:48.826918 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.826871    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"]
Jul 26 16:14:48.831310 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.831294    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:14:48.833404 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.833380    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-7cc89d\""
Jul 26 16:14:48.833527 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.833408    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-7cc89d-dockercfg-ngzlm\""
Jul 26 16:14:48.842071 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.842046    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"]
Jul 26 16:14:48.860885 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.860856    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"]
Jul 26 16:14:48.864103 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.864082    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:14:48.875455 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.875434    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"]
Jul 26 16:14:48.924433 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.924406    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/286e10bf-ad63-48fc-84fc-a42ad98c2996-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x\" (UID: \"286e10bf-ad63-48fc-84fc-a42ad98c2996\") " pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:14:48.924593 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:48.924573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22rpj\" (UniqueName: \"kubernetes.io/projected/286e10bf-ad63-48fc-84fc-a42ad98c2996-kube-api-access-22rpj\") pod \"payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x\" (UID: \"286e10bf-ad63-48fc-84fc-a42ad98c2996\") " pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:14:49.025344 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.025305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22rpj\" (UniqueName: \"kubernetes.io/projected/286e10bf-ad63-48fc-84fc-a42ad98c2996-kube-api-access-22rpj\") pod \"payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x\" (UID: \"286e10bf-ad63-48fc-84fc-a42ad98c2996\") " pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:14:49.025541 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.025508    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/286e10bf-ad63-48fc-84fc-a42ad98c2996-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x\" (UID: \"286e10bf-ad63-48fc-84fc-a42ad98c2996\") " pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:14:49.025709 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.025592    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cftwc\" (UniqueName: \"kubernetes.io/projected/49318478-e13a-4e5c-8a0c-90c720df4583-kube-api-access-cftwc\") pod \"payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49\" (UID: \"49318478-e13a-4e5c-8a0c-90c720df4583\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:14:49.025709 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.025648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49318478-e13a-4e5c-8a0c-90c720df4583-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49\" (UID: \"49318478-e13a-4e5c-8a0c-90c720df4583\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:14:49.026383 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.026362    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/286e10bf-ad63-48fc-84fc-a42ad98c2996-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x\" (UID: \"286e10bf-ad63-48fc-84fc-a42ad98c2996\") " pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:14:49.035383 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.035360    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22rpj\" (UniqueName: \"kubernetes.io/projected/286e10bf-ad63-48fc-84fc-a42ad98c2996-kube-api-access-22rpj\") pod \"payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x\" (UID: \"286e10bf-ad63-48fc-84fc-a42ad98c2996\") " pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:14:49.126804 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.126710    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49318478-e13a-4e5c-8a0c-90c720df4583-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49\" (UID: \"49318478-e13a-4e5c-8a0c-90c720df4583\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:14:49.126960 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.126815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cftwc\" (UniqueName: \"kubernetes.io/projected/49318478-e13a-4e5c-8a0c-90c720df4583-kube-api-access-cftwc\") pod \"payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49\" (UID: \"49318478-e13a-4e5c-8a0c-90c720df4583\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:14:49.127580 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.127557    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49318478-e13a-4e5c-8a0c-90c720df4583-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49\" (UID: \"49318478-e13a-4e5c-8a0c-90c720df4583\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:14:49.134474 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.134447    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cftwc\" (UniqueName: \"kubernetes.io/projected/49318478-e13a-4e5c-8a0c-90c720df4583-kube-api-access-cftwc\") pod \"payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49\" (UID: \"49318478-e13a-4e5c-8a0c-90c720df4583\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:14:49.142505 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.142488    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:14:49.176317 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.176270    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:14:49.285966 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.285937    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"]
Jul 26 16:14:49.289736 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:14:49.289706    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286e10bf_ad63_48fc_84fc_a42ad98c2996.slice/crio-4c22c0683f995fca14a24ba0ce63317d22a464bfc0f9cb8cdd99279b3a6095cc WatchSource:0}: Error finding container 4c22c0683f995fca14a24ba0ce63317d22a464bfc0f9cb8cdd99279b3a6095cc: Status 404 returned error can't find the container with id 4c22c0683f995fca14a24ba0ce63317d22a464bfc0f9cb8cdd99279b3a6095cc
Jul 26 16:14:49.324964 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.324940    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"]
Jul 26 16:14:49.328043 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:14:49.328020    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49318478_e13a_4e5c_8a0c_90c720df4583.slice/crio-0ad11182ff1c04c969b9e508567478fa16e0e85cb4fef7941acb39669a9457cf WatchSource:0}: Error finding container 0ad11182ff1c04c969b9e508567478fa16e0e85cb4fef7941acb39669a9457cf: Status 404 returned error can't find the container with id 0ad11182ff1c04c969b9e508567478fa16e0e85cb4fef7941acb39669a9457cf
Jul 26 16:14:49.517335 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.517299    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7" event={"ID":"b73caccd-47ad-4840-ad29-c5ce450da310","Type":"ContainerStarted","Data":"80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6"}
Jul 26 16:14:49.517534 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.517360    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7" event={"ID":"b73caccd-47ad-4840-ad29-c5ce450da310","Type":"ContainerStarted","Data":"207768c518fd51274160361a9d7a985b37fc66b800e4026c0541d543b0e38a21"}
Jul 26 16:14:49.517534 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.517524    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:14:49.519704 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.519674    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49" event={"ID":"49318478-e13a-4e5c-8a0c-90c720df4583","Type":"ContainerStarted","Data":"66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19"}
Jul 26 16:14:49.519873 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.519723    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49" event={"ID":"49318478-e13a-4e5c-8a0c-90c720df4583","Type":"ContainerStarted","Data":"0ad11182ff1c04c969b9e508567478fa16e0e85cb4fef7941acb39669a9457cf"}
Jul 26 16:14:49.519873 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.519747    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:14:49.522408 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.522382    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x" event={"ID":"286e10bf-ad63-48fc-84fc-a42ad98c2996","Type":"ContainerStarted","Data":"89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7"}
Jul 26 16:14:49.522527 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.522442    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x" event={"ID":"286e10bf-ad63-48fc-84fc-a42ad98c2996","Type":"ContainerStarted","Data":"4c22c0683f995fca14a24ba0ce63317d22a464bfc0f9cb8cdd99279b3a6095cc"}
Jul 26 16:14:49.522527 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.522471    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:14:49.535236 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.535194    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7" podStartSLOduration=1.189832241 podStartE2EDuration="1.53517919s" totalImagesPullingTime="345.346949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:14:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:14:49.533825574 +0000 UTC m=+2205.033215413" watchObservedRunningTime="2026-07-26 16:14:49.53517919 +0000 UTC m=+2205.034569028"
Jul 26 16:14:49.550208 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.550157    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x" podStartSLOduration=1.550144073 podStartE2EDuration="1.550144073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:14:49.548535459 +0000 UTC m=+2205.047925296" watchObservedRunningTime="2026-07-26 16:14:49.550144073 +0000 UTC m=+2205.049533935"
Jul 26 16:14:49.564628 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:49.564588    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49" podStartSLOduration=1.564576825 podStartE2EDuration="1.564576825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:14:49.563042644 +0000 UTC m=+2205.062432472" watchObservedRunningTime="2026-07-26 16:14:49.564576825 +0000 UTC m=+2205.063966666"
Jul 26 16:14:52.478779 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:52.478749    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:14:53.482406 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:53.482378    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:14:55.532305 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:14:55.532280    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:15:00.127427 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.127394    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws"]
Jul 26 16:15:00.132075 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.132053    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws"
Jul 26 16:15:00.134221 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.134202    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:15:00.139228 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.138970    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws"]
Jul 26 16:15:00.142686 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.142666    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws"
Jul 26 16:15:00.289136 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.289107    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws"]
Jul 26 16:15:00.292394 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:15:00.292366    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c2a632_244c_4438_86e8_17f3272d7d15.slice/crio-ae64c1a2009328dbb500e11f8a18fbfe79b71d172c32c103ab5b24673fa64ddd WatchSource:0}: Error finding container ae64c1a2009328dbb500e11f8a18fbfe79b71d172c32c103ab5b24673fa64ddd: Status 404 returned error can't find the container with id ae64c1a2009328dbb500e11f8a18fbfe79b71d172c32c103ab5b24673fa64ddd
Jul 26 16:15:00.392853 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.392826    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"]
Jul 26 16:15:00.397378 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.397362    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"
Jul 26 16:15:00.402082 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.402061    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"]
Jul 26 16:15:00.519179 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.519153    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52244c37-24f7-4fd8-8aa0-bbd22a868e29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6\" (UID: \"52244c37-24f7-4fd8-8aa0-bbd22a868e29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"
Jul 26 16:15:00.529259 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.529170    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:15:00.529571 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.529541    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:15:00.574309 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.574197    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws" event={"ID":"c1c2a632-244c-4438-86e8-17f3272d7d15","Type":"ContainerStarted","Data":"ae64c1a2009328dbb500e11f8a18fbfe79b71d172c32c103ab5b24673fa64ddd"}
Jul 26 16:15:00.620610 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.620581    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52244c37-24f7-4fd8-8aa0-bbd22a868e29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6\" (UID: \"52244c37-24f7-4fd8-8aa0-bbd22a868e29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"
Jul 26 16:15:00.621264 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.621246    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52244c37-24f7-4fd8-8aa0-bbd22a868e29-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6\" (UID: \"52244c37-24f7-4fd8-8aa0-bbd22a868e29\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"
Jul 26 16:15:00.707532 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:00.707474    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"
Jul 26 16:15:01.039419 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:01.039236    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"]
Jul 26 16:15:01.042137 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:15:01.042108    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52244c37_24f7_4fd8_8aa0_bbd22a868e29.slice/crio-072175166e369be883ceef8d7ddc55cdaa7917f4cf4e1f9b54f24028287c5229 WatchSource:0}: Error finding container 072175166e369be883ceef8d7ddc55cdaa7917f4cf4e1f9b54f24028287c5229: Status 404 returned error can't find the container with id 072175166e369be883ceef8d7ddc55cdaa7917f4cf4e1f9b54f24028287c5229
Jul 26 16:15:01.580023 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:01.579975    2560 generic.go:356] "Generic (PLEG): container finished" podID="c1c2a632-244c-4438-86e8-17f3272d7d15" containerID="cfcdf21f9bef5405c0eae6680fc1671a4590fdcb0bc36962ddcdf408ecfa0460" exitCode=0
Jul 26 16:15:01.580450 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:01.580058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws" event={"ID":"c1c2a632-244c-4438-86e8-17f3272d7d15","Type":"ContainerDied","Data":"cfcdf21f9bef5405c0eae6680fc1671a4590fdcb0bc36962ddcdf408ecfa0460"}
Jul 26 16:15:01.581546 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:01.581523    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6" event={"ID":"52244c37-24f7-4fd8-8aa0-bbd22a868e29","Type":"ContainerStarted","Data":"ec425ddb18e354428cdce726226a41707583b76d1932647488ff7bf16cb5c2f8"}
Jul 26 16:15:01.581641 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:01.581573    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6" event={"ID":"52244c37-24f7-4fd8-8aa0-bbd22a868e29","Type":"ContainerStarted","Data":"072175166e369be883ceef8d7ddc55cdaa7917f4cf4e1f9b54f24028287c5229"}
Jul 26 16:15:01.606464 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:01.606375    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6" podStartSLOduration=1.606362643 podStartE2EDuration="1.606362643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:15:01.605908692 +0000 UTC m=+2217.105298528" watchObservedRunningTime="2026-07-26 16:15:01.606362643 +0000 UTC m=+2217.105752498"
Jul 26 16:15:02.588323 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:02.588288    2560 generic.go:356] "Generic (PLEG): container finished" podID="52244c37-24f7-4fd8-8aa0-bbd22a868e29" containerID="ec425ddb18e354428cdce726226a41707583b76d1932647488ff7bf16cb5c2f8" exitCode=0
Jul 26 16:15:02.588699 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:02.588371    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6" event={"ID":"52244c37-24f7-4fd8-8aa0-bbd22a868e29","Type":"ContainerDied","Data":"ec425ddb18e354428cdce726226a41707583b76d1932647488ff7bf16cb5c2f8"}
Jul 26 16:15:02.720084 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:02.720061    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws"
Jul 26 16:15:03.594101 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:03.594034    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws"
Jul 26 16:15:03.594101 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:03.594072    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws" event={"ID":"c1c2a632-244c-4438-86e8-17f3272d7d15","Type":"ContainerDied","Data":"ae64c1a2009328dbb500e11f8a18fbfe79b71d172c32c103ab5b24673fa64ddd"}
Jul 26 16:15:03.594534 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:03.594113    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae64c1a2009328dbb500e11f8a18fbfe79b71d172c32c103ab5b24673fa64ddd"
Jul 26 16:15:03.724771 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:03.724737    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"
Jul 26 16:15:03.741565 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:03.741541    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52244c37-24f7-4fd8-8aa0-bbd22a868e29-maas-api-service-ca\") pod \"52244c37-24f7-4fd8-8aa0-bbd22a868e29\" (UID: \"52244c37-24f7-4fd8-8aa0-bbd22a868e29\") "
Jul 26 16:15:03.741842 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:03.741819    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52244c37-24f7-4fd8-8aa0-bbd22a868e29-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52244c37-24f7-4fd8-8aa0-bbd22a868e29" (UID: "52244c37-24f7-4fd8-8aa0-bbd22a868e29"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:15:03.842867 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:03.842846    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52244c37-24f7-4fd8-8aa0-bbd22a868e29-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:04.599459 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:04.599428    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"
Jul 26 16:15:04.599883 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:04.599470    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6" event={"ID":"52244c37-24f7-4fd8-8aa0-bbd22a868e29","Type":"ContainerDied","Data":"072175166e369be883ceef8d7ddc55cdaa7917f4cf4e1f9b54f24028287c5229"}
Jul 26 16:15:04.599883 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:04.599524    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="072175166e369be883ceef8d7ddc55cdaa7917f4cf4e1f9b54f24028287c5229"
Jul 26 16:15:05.249610 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:05.249581    2560 scope.go:117] "RemoveContainer" containerID="7c3f1d753f6b1b74c1e9dfd9fe5861bc06d74a368efd8d36947006bbd9ed0a06"
Jul 26 16:15:15.815705 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:15.815667    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"]
Jul 26 16:15:15.818082 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:15.816003    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc" podUID="3ff62ae5-ee2c-451d-ab61-88a15cb6f388" containerName="maas-api" containerID="cri-o://41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162" gracePeriod=30
Jul 26 16:15:15.866441 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:15.866406    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"]
Jul 26 16:15:15.866733 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:15.866671    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt" podUID="d3f99705-5e2b-45b0-a6d3-791e8787fd2e" containerName="payload-processing" containerID="cri-o://f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b" gracePeriod=30
Jul 26 16:15:15.874365 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:15.874331    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"]
Jul 26 16:15:15.874769 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:15.874582    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf" podUID="4b10f97e-76a4-4201-b123-9e296f6609d7" containerName="payload-pre-processing" containerID="cri-o://a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510" gracePeriod=30
Jul 26 16:15:16.171952 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.171927    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:15:16.175045 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.175024    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:15:16.178253 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.178230    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:15:16.236391 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.236363    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98jh7\" (UniqueName: \"kubernetes.io/projected/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-kube-api-access-98jh7\") pod \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") "
Jul 26 16:15:16.236554 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.236399    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-maas-api-tls\") pod \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") "
Jul 26 16:15:16.236554 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.236418    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-plugins-config-volume\") pod \"d3f99705-5e2b-45b0-a6d3-791e8787fd2e\" (UID: \"d3f99705-5e2b-45b0-a6d3-791e8787fd2e\") "
Jul 26 16:15:16.236554 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.236471    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b10f97e-76a4-4201-b123-9e296f6609d7-plugins-config-volume\") pod \"4b10f97e-76a4-4201-b123-9e296f6609d7\" (UID: \"4b10f97e-76a4-4201-b123-9e296f6609d7\") "
Jul 26 16:15:16.236554 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.236488    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh4wx\" (UniqueName: \"kubernetes.io/projected/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-kube-api-access-lh4wx\") pod \"d3f99705-5e2b-45b0-a6d3-791e8787fd2e\" (UID: \"d3f99705-5e2b-45b0-a6d3-791e8787fd2e\") "
Jul 26 16:15:16.236554 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.236514    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-openshift-service-ca\") pod \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\" (UID: \"3ff62ae5-ee2c-451d-ab61-88a15cb6f388\") "
Jul 26 16:15:16.236554 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.236544    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z6g6\" (UniqueName: \"kubernetes.io/projected/4b10f97e-76a4-4201-b123-9e296f6609d7-kube-api-access-6z6g6\") pod \"4b10f97e-76a4-4201-b123-9e296f6609d7\" (UID: \"4b10f97e-76a4-4201-b123-9e296f6609d7\") "
Jul 26 16:15:16.236907 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.236860    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b10f97e-76a4-4201-b123-9e296f6609d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b10f97e-76a4-4201-b123-9e296f6609d7" (UID: "4b10f97e-76a4-4201-b123-9e296f6609d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:15:16.237189 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.237156    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3ff62ae5-ee2c-451d-ab61-88a15cb6f388" (UID: "3ff62ae5-ee2c-451d-ab61-88a15cb6f388"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:15:16.237306 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.237180    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d3f99705-5e2b-45b0-a6d3-791e8787fd2e" (UID: "d3f99705-5e2b-45b0-a6d3-791e8787fd2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:15:16.238743 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.238719    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b10f97e-76a4-4201-b123-9e296f6609d7-kube-api-access-6z6g6" (OuterVolumeSpecName: "kube-api-access-6z6g6") pod "4b10f97e-76a4-4201-b123-9e296f6609d7" (UID: "4b10f97e-76a4-4201-b123-9e296f6609d7"). InnerVolumeSpecName "kube-api-access-6z6g6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:15:16.238859 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.238718    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-kube-api-access-98jh7" (OuterVolumeSpecName: "kube-api-access-98jh7") pod "3ff62ae5-ee2c-451d-ab61-88a15cb6f388" (UID: "3ff62ae5-ee2c-451d-ab61-88a15cb6f388"). InnerVolumeSpecName "kube-api-access-98jh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:15:16.238932 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.238908    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3ff62ae5-ee2c-451d-ab61-88a15cb6f388" (UID: "3ff62ae5-ee2c-451d-ab61-88a15cb6f388"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:15:16.239265 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.239247    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-kube-api-access-lh4wx" (OuterVolumeSpecName: "kube-api-access-lh4wx") pod "d3f99705-5e2b-45b0-a6d3-791e8787fd2e" (UID: "d3f99705-5e2b-45b0-a6d3-791e8787fd2e"). InnerVolumeSpecName "kube-api-access-lh4wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:15:16.337396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.337373    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:16.337396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.337395    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z6g6\" (UniqueName: \"kubernetes.io/projected/4b10f97e-76a4-4201-b123-9e296f6609d7-kube-api-access-6z6g6\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:16.337528 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.337405    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98jh7\" (UniqueName: \"kubernetes.io/projected/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-kube-api-access-98jh7\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:16.337528 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.337414    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff62ae5-ee2c-451d-ab61-88a15cb6f388-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:16.337528 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.337423    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:16.337528 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.337431    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b10f97e-76a4-4201-b123-9e296f6609d7-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:16.337528 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.337439    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh4wx\" (UniqueName: \"kubernetes.io/projected/d3f99705-5e2b-45b0-a6d3-791e8787fd2e-kube-api-access-lh4wx\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:16.659259 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.659222    2560 generic.go:356] "Generic (PLEG): container finished" podID="4b10f97e-76a4-4201-b123-9e296f6609d7" containerID="a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510" exitCode=0
Jul 26 16:15:16.659427 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.659265    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"
Jul 26 16:15:16.659427 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.659302    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf" event={"ID":"4b10f97e-76a4-4201-b123-9e296f6609d7","Type":"ContainerDied","Data":"a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510"}
Jul 26 16:15:16.659427 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.659356    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf" event={"ID":"4b10f97e-76a4-4201-b123-9e296f6609d7","Type":"ContainerDied","Data":"d57d1b6fff0ff672d1f56849e1b61710c1e3afd85e50983ef9f5ce1633571500"}
Jul 26 16:15:16.659427 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.659380    2560 scope.go:117] "RemoveContainer" containerID="a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510"
Jul 26 16:15:16.660725 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.660689    2560 generic.go:356] "Generic (PLEG): container finished" podID="3ff62ae5-ee2c-451d-ab61-88a15cb6f388" containerID="41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162" exitCode=0
Jul 26 16:15:16.660839 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.660734    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"
Jul 26 16:15:16.660839 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.660800    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc" event={"ID":"3ff62ae5-ee2c-451d-ab61-88a15cb6f388","Type":"ContainerDied","Data":"41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162"}
Jul 26 16:15:16.661036 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.660851    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc" event={"ID":"3ff62ae5-ee2c-451d-ab61-88a15cb6f388","Type":"ContainerDied","Data":"2ad38849dadf6a9c79079eaf2f71768d54c25673c6eed2cd27ba9878b0784d34"}
Jul 26 16:15:16.662254 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.662230    2560 generic.go:356] "Generic (PLEG): container finished" podID="d3f99705-5e2b-45b0-a6d3-791e8787fd2e" containerID="f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b" exitCode=0
Jul 26 16:15:16.662254 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.662268    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt" event={"ID":"d3f99705-5e2b-45b0-a6d3-791e8787fd2e","Type":"ContainerDied","Data":"f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b"}
Jul 26 16:15:16.662428 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.662303    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt" event={"ID":"d3f99705-5e2b-45b0-a6d3-791e8787fd2e","Type":"ContainerDied","Data":"c473cf13eb3d1530948b06d0c7225512de01ab77fa611541ad28f35d58e3175b"}
Jul 26 16:15:16.662428 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.662270    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"
Jul 26 16:15:16.668642 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.668618    2560 scope.go:117] "RemoveContainer" containerID="a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510"
Jul 26 16:15:16.668946 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:15:16.668923    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510\": container with ID starting with a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510 not found: ID does not exist" containerID="a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510"
Jul 26 16:15:16.669047 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.668952    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510"} err="failed to get container status \"a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510\": rpc error: code = NotFound desc = could not find container \"a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510\": container with ID starting with a4cbb978853906faf44681d670f3fc37425e4fd9092726f9b983cace0aaa8510 not found: ID does not exist"
Jul 26 16:15:16.669047 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.668967    2560 scope.go:117] "RemoveContainer" containerID="41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162"
Jul 26 16:15:16.677653 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.677636    2560 scope.go:117] "RemoveContainer" containerID="41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162"
Jul 26 16:15:16.677922 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:15:16.677881    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162\": container with ID starting with 41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162 not found: ID does not exist" containerID="41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162"
Jul 26 16:15:16.677977 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.677929    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162"} err="failed to get container status \"41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162\": rpc error: code = NotFound desc = could not find container \"41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162\": container with ID starting with 41857287d397a76f346b62329439422cca6cb965e12a37be8a27f7c6a4c97162 not found: ID does not exist"
Jul 26 16:15:16.677977 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.677944    2560 scope.go:117] "RemoveContainer" containerID="f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b"
Jul 26 16:15:16.684954 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.684934    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"]
Jul 26 16:15:16.685460 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.685445    2560 scope.go:117] "RemoveContainer" containerID="f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b"
Jul 26 16:15:16.685709 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:15:16.685690    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b\": container with ID starting with f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b not found: ID does not exist" containerID="f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b"
Jul 26 16:15:16.685767 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.685718    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b"} err="failed to get container status \"f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b\": rpc error: code = NotFound desc = could not find container \"f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b\": container with ID starting with f95106e0182d4f0be20a582253fc90589ae520df6d9a93e11841d482563c236b not found: ID does not exist"
Jul 26 16:15:16.691042 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.691020    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4b0d96-8654dd44bd-bl8pf"]
Jul 26 16:15:16.700486 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.700464    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"]
Jul 26 16:15:16.706462 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.706443    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4b0d96-6ddd4c79cd-p5jkt"]
Jul 26 16:15:16.715750 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.715731    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"]
Jul 26 16:15:16.719232 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:16.719211    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4b0d96-675db488b4-9qsgc"]
Jul 26 16:15:17.139621 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:17.139588    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ff62ae5-ee2c-451d-ab61-88a15cb6f388" path="/var/lib/kubelet/pods/3ff62ae5-ee2c-451d-ab61-88a15cb6f388/volumes"
Jul 26 16:15:17.140030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:17.140013    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b10f97e-76a4-4201-b123-9e296f6609d7" path="/var/lib/kubelet/pods/4b10f97e-76a4-4201-b123-9e296f6609d7/volumes"
Jul 26 16:15:17.140379 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:17.140366    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3f99705-5e2b-45b0-a6d3-791e8787fd2e" path="/var/lib/kubelet/pods/d3f99705-5e2b-45b0-a6d3-791e8787fd2e/volumes"
Jul 26 16:15:20.466150 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:20.466118    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"]
Jul 26 16:15:20.468540 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:20.468516    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-4b0d96-sh5t6"]
Jul 26 16:15:21.139864 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:21.139832    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52244c37-24f7-4fd8-8aa0-bbd22a868e29" path="/var/lib/kubelet/pods/52244c37-24f7-4fd8-8aa0-bbd22a868e29/volumes"
Jul 26 16:15:42.858121 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:42.858077    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"]
Jul 26 16:15:42.858546 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:42.858312    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7" podUID="b73caccd-47ad-4840-ad29-c5ce450da310" containerName="maas-api" containerID="cri-o://80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6" gracePeriod=30
Jul 26 16:15:42.874301 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:42.874277    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws"]
Jul 26 16:15:42.880695 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:42.880670    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-7cc89d-29751375-tqrws"]
Jul 26 16:15:42.909732 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:42.909705    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"]
Jul 26 16:15:42.919410 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:42.919343    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x" podUID="286e10bf-ad63-48fc-84fc-a42ad98c2996" containerName="payload-processing" containerID="cri-o://89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7" gracePeriod=30
Jul 26 16:15:42.936470 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:42.935449    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"]
Jul 26 16:15:42.936470 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:42.935926    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49" podUID="49318478-e13a-4e5c-8a0c-90c720df4583" containerName="payload-pre-processing" containerID="cri-o://66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19" gracePeriod=30
Jul 26 16:15:43.141775 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.141746    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1c2a632-244c-4438-86e8-17f3272d7d15" path="/var/lib/kubelet/pods/c1c2a632-244c-4438-86e8-17f3272d7d15/volumes"
Jul 26 16:15:43.205090 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.205066    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:15:43.209960 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.209938    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:15:43.220621 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.220603    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:15:43.350075 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.350045    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73caccd-47ad-4840-ad29-c5ce450da310-maas-api-tls\") pod \"b73caccd-47ad-4840-ad29-c5ce450da310\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") "
Jul 26 16:15:43.350234 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.350100    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn9bt\" (UniqueName: \"kubernetes.io/projected/b73caccd-47ad-4840-ad29-c5ce450da310-kube-api-access-pn9bt\") pod \"b73caccd-47ad-4840-ad29-c5ce450da310\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") "
Jul 26 16:15:43.350234 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.350119    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73caccd-47ad-4840-ad29-c5ce450da310-openshift-service-ca\") pod \"b73caccd-47ad-4840-ad29-c5ce450da310\" (UID: \"b73caccd-47ad-4840-ad29-c5ce450da310\") "
Jul 26 16:15:43.350234 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.350169    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22rpj\" (UniqueName: \"kubernetes.io/projected/286e10bf-ad63-48fc-84fc-a42ad98c2996-kube-api-access-22rpj\") pod \"286e10bf-ad63-48fc-84fc-a42ad98c2996\" (UID: \"286e10bf-ad63-48fc-84fc-a42ad98c2996\") "
Jul 26 16:15:43.350234 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.350193    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/286e10bf-ad63-48fc-84fc-a42ad98c2996-plugins-config-volume\") pod \"286e10bf-ad63-48fc-84fc-a42ad98c2996\" (UID: \"286e10bf-ad63-48fc-84fc-a42ad98c2996\") "
Jul 26 16:15:43.350234 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.350209    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cftwc\" (UniqueName: \"kubernetes.io/projected/49318478-e13a-4e5c-8a0c-90c720df4583-kube-api-access-cftwc\") pod \"49318478-e13a-4e5c-8a0c-90c720df4583\" (UID: \"49318478-e13a-4e5c-8a0c-90c720df4583\") "
Jul 26 16:15:43.350518 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.350244    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49318478-e13a-4e5c-8a0c-90c720df4583-plugins-config-volume\") pod \"49318478-e13a-4e5c-8a0c-90c720df4583\" (UID: \"49318478-e13a-4e5c-8a0c-90c720df4583\") "
Jul 26 16:15:43.350731 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.350601    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b73caccd-47ad-4840-ad29-c5ce450da310-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b73caccd-47ad-4840-ad29-c5ce450da310" (UID: "b73caccd-47ad-4840-ad29-c5ce450da310"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:15:43.350824 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.350721    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/286e10bf-ad63-48fc-84fc-a42ad98c2996-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "286e10bf-ad63-48fc-84fc-a42ad98c2996" (UID: "286e10bf-ad63-48fc-84fc-a42ad98c2996"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:15:43.350824 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.350759    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49318478-e13a-4e5c-8a0c-90c720df4583-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49318478-e13a-4e5c-8a0c-90c720df4583" (UID: "49318478-e13a-4e5c-8a0c-90c720df4583"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:15:43.352335 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.352312    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49318478-e13a-4e5c-8a0c-90c720df4583-kube-api-access-cftwc" (OuterVolumeSpecName: "kube-api-access-cftwc") pod "49318478-e13a-4e5c-8a0c-90c720df4583" (UID: "49318478-e13a-4e5c-8a0c-90c720df4583"). InnerVolumeSpecName "kube-api-access-cftwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:15:43.352526 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.352354    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/286e10bf-ad63-48fc-84fc-a42ad98c2996-kube-api-access-22rpj" (OuterVolumeSpecName: "kube-api-access-22rpj") pod "286e10bf-ad63-48fc-84fc-a42ad98c2996" (UID: "286e10bf-ad63-48fc-84fc-a42ad98c2996"). InnerVolumeSpecName "kube-api-access-22rpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:15:43.352594 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.352524    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b73caccd-47ad-4840-ad29-c5ce450da310-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b73caccd-47ad-4840-ad29-c5ce450da310" (UID: "b73caccd-47ad-4840-ad29-c5ce450da310"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:15:43.352594 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.352551    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b73caccd-47ad-4840-ad29-c5ce450da310-kube-api-access-pn9bt" (OuterVolumeSpecName: "kube-api-access-pn9bt") pod "b73caccd-47ad-4840-ad29-c5ce450da310" (UID: "b73caccd-47ad-4840-ad29-c5ce450da310"). InnerVolumeSpecName "kube-api-access-pn9bt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:15:43.451811 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.451780    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73caccd-47ad-4840-ad29-c5ce450da310-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:43.451811 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.451806    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22rpj\" (UniqueName: \"kubernetes.io/projected/286e10bf-ad63-48fc-84fc-a42ad98c2996-kube-api-access-22rpj\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:43.451811 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.451817    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/286e10bf-ad63-48fc-84fc-a42ad98c2996-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:43.452056 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.451826    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cftwc\" (UniqueName: \"kubernetes.io/projected/49318478-e13a-4e5c-8a0c-90c720df4583-kube-api-access-cftwc\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:43.452056 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.451837    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49318478-e13a-4e5c-8a0c-90c720df4583-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:43.452056 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.451846    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73caccd-47ad-4840-ad29-c5ce450da310-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:43.452056 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.451854    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn9bt\" (UniqueName: \"kubernetes.io/projected/b73caccd-47ad-4840-ad29-c5ce450da310-kube-api-access-pn9bt\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:15:43.792021 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.791928    2560 generic.go:356] "Generic (PLEG): container finished" podID="49318478-e13a-4e5c-8a0c-90c720df4583" containerID="66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19" exitCode=0
Jul 26 16:15:43.792021 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.791981    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"
Jul 26 16:15:43.792248 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.792024    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49" event={"ID":"49318478-e13a-4e5c-8a0c-90c720df4583","Type":"ContainerDied","Data":"66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19"}
Jul 26 16:15:43.792248 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.792070    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49" event={"ID":"49318478-e13a-4e5c-8a0c-90c720df4583","Type":"ContainerDied","Data":"0ad11182ff1c04c969b9e508567478fa16e0e85cb4fef7941acb39669a9457cf"}
Jul 26 16:15:43.792248 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.792094    2560 scope.go:117] "RemoveContainer" containerID="66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19"
Jul 26 16:15:43.793339 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.793314    2560 generic.go:356] "Generic (PLEG): container finished" podID="286e10bf-ad63-48fc-84fc-a42ad98c2996" containerID="89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7" exitCode=0
Jul 26 16:15:43.793446 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.793342    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x" event={"ID":"286e10bf-ad63-48fc-84fc-a42ad98c2996","Type":"ContainerDied","Data":"89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7"}
Jul 26 16:15:43.793446 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.793376    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"
Jul 26 16:15:43.793446 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.793386    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x" event={"ID":"286e10bf-ad63-48fc-84fc-a42ad98c2996","Type":"ContainerDied","Data":"4c22c0683f995fca14a24ba0ce63317d22a464bfc0f9cb8cdd99279b3a6095cc"}
Jul 26 16:15:43.794830 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.794808    2560 generic.go:356] "Generic (PLEG): container finished" podID="b73caccd-47ad-4840-ad29-c5ce450da310" containerID="80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6" exitCode=0
Jul 26 16:15:43.794966 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.794843    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7" event={"ID":"b73caccd-47ad-4840-ad29-c5ce450da310","Type":"ContainerDied","Data":"80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6"}
Jul 26 16:15:43.794966 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.794854    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"
Jul 26 16:15:43.794966 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.794881    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7" event={"ID":"b73caccd-47ad-4840-ad29-c5ce450da310","Type":"ContainerDied","Data":"207768c518fd51274160361a9d7a985b37fc66b800e4026c0541d543b0e38a21"}
Jul 26 16:15:43.802909 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.802870    2560 scope.go:117] "RemoveContainer" containerID="66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19"
Jul 26 16:15:43.803194 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:15:43.803175    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19\": container with ID starting with 66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19 not found: ID does not exist" containerID="66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19"
Jul 26 16:15:43.803276 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.803206    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19"} err="failed to get container status \"66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19\": rpc error: code = NotFound desc = could not find container \"66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19\": container with ID starting with 66db5451683dfda9836bf4e8016a3764d27173439429fe2ed84e3452f19f5e19 not found: ID does not exist"
Jul 26 16:15:43.803276 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.803228    2560 scope.go:117] "RemoveContainer" containerID="89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7"
Jul 26 16:15:43.810642 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.810626    2560 scope.go:117] "RemoveContainer" containerID="89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7"
Jul 26 16:15:43.810931 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:15:43.810908    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7\": container with ID starting with 89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7 not found: ID does not exist" containerID="89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7"
Jul 26 16:15:43.811018 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.810936    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7"} err="failed to get container status \"89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7\": rpc error: code = NotFound desc = could not find container \"89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7\": container with ID starting with 89108652bfe93e9ab253b36367e6a44ac57852ce4df5a06904daa1ad2676e7d7 not found: ID does not exist"
Jul 26 16:15:43.811018 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.810953    2560 scope.go:117] "RemoveContainer" containerID="80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6"
Jul 26 16:15:43.814246 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.814228    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"]
Jul 26 16:15:43.817553 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.817529    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7cc89d-55bc6fb564-bmw49"]
Jul 26 16:15:43.818826 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.818809    2560 scope.go:117] "RemoveContainer" containerID="80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6"
Jul 26 16:15:43.819103 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:15:43.819085    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6\": container with ID starting with 80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6 not found: ID does not exist" containerID="80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6"
Jul 26 16:15:43.819153 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.819109    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6"} err="failed to get container status \"80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6\": rpc error: code = NotFound desc = could not find container \"80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6\": container with ID starting with 80f7569155bd3758afc1b9251b81d79d3a884d95f852bafe6d0e89ebe98c07c6 not found: ID does not exist"
Jul 26 16:15:43.827023 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.827004    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"]
Jul 26 16:15:43.829786 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.829767    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7cc89d-57fbb849f4-f8d9x"]
Jul 26 16:15:43.839333 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.839314    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"]
Jul 26 16:15:43.845258 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:43.845239    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7cc89d-6584f998f8-mpwq7"]
Jul 26 16:15:45.138768 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:45.138734    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="286e10bf-ad63-48fc-84fc-a42ad98c2996" path="/var/lib/kubelet/pods/286e10bf-ad63-48fc-84fc-a42ad98c2996/volumes"
Jul 26 16:15:45.139137 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:45.139102    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49318478-e13a-4e5c-8a0c-90c720df4583" path="/var/lib/kubelet/pods/49318478-e13a-4e5c-8a0c-90c720df4583/volumes"
Jul 26 16:15:45.139434 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:45.139421    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b73caccd-47ad-4840-ad29-c5ce450da310" path="/var/lib/kubelet/pods/b73caccd-47ad-4840-ad29-c5ce450da310/volumes"
Jul 26 16:15:57.529197 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529161    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"]
Jul 26 16:15:57.529653 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529610    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="286e10bf-ad63-48fc-84fc-a42ad98c2996" containerName="payload-processing"
Jul 26 16:15:57.529653 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529628    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="286e10bf-ad63-48fc-84fc-a42ad98c2996" containerName="payload-processing"
Jul 26 16:15:57.529653 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529644    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52244c37-24f7-4fd8-8aa0-bbd22a868e29" containerName="revoke-keys"
Jul 26 16:15:57.529653 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529653    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="52244c37-24f7-4fd8-8aa0-bbd22a868e29" containerName="revoke-keys"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529671    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b73caccd-47ad-4840-ad29-c5ce450da310" containerName="maas-api"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529679    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b73caccd-47ad-4840-ad29-c5ce450da310" containerName="maas-api"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529697    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b10f97e-76a4-4201-b123-9e296f6609d7" containerName="payload-pre-processing"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529706    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b10f97e-76a4-4201-b123-9e296f6609d7" containerName="payload-pre-processing"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529723    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49318478-e13a-4e5c-8a0c-90c720df4583" containerName="payload-pre-processing"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529731    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="49318478-e13a-4e5c-8a0c-90c720df4583" containerName="payload-pre-processing"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529743    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c2a632-244c-4438-86e8-17f3272d7d15" containerName="cleanup"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529751    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c2a632-244c-4438-86e8-17f3272d7d15" containerName="cleanup"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529763    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f99705-5e2b-45b0-a6d3-791e8787fd2e" containerName="payload-processing"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529771    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f99705-5e2b-45b0-a6d3-791e8787fd2e" containerName="payload-processing"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529781    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff62ae5-ee2c-451d-ab61-88a15cb6f388" containerName="maas-api"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529790    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff62ae5-ee2c-451d-ab61-88a15cb6f388" containerName="maas-api"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529864    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3f99705-5e2b-45b0-a6d3-791e8787fd2e" containerName="payload-processing"
Jul 26 16:15:57.529871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529876    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b10f97e-76a4-4201-b123-9e296f6609d7" containerName="payload-pre-processing"
Jul 26 16:15:57.530561 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529905    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1c2a632-244c-4438-86e8-17f3272d7d15" containerName="cleanup"
Jul 26 16:15:57.530561 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529915    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="286e10bf-ad63-48fc-84fc-a42ad98c2996" containerName="payload-processing"
Jul 26 16:15:57.530561 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529923    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="49318478-e13a-4e5c-8a0c-90c720df4583" containerName="payload-pre-processing"
Jul 26 16:15:57.530561 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529935    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff62ae5-ee2c-451d-ab61-88a15cb6f388" containerName="maas-api"
Jul 26 16:15:57.530561 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529943    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="52244c37-24f7-4fd8-8aa0-bbd22a868e29" containerName="revoke-keys"
Jul 26 16:15:57.530561 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.529954    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b73caccd-47ad-4840-ad29-c5ce450da310" containerName="maas-api"
Jul 26 16:15:57.534734 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.534711    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.537849 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.537816    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-aadfb2-serving-cert\""
Jul 26 16:15:57.558859 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.558822    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"]
Jul 26 16:15:57.672439 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.672415    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13d5469-5380-4b13-97dc-9a5ff72adcd6-maas-api-tls\") pod \"maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.672570 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.672475    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s67vq\" (UniqueName: \"kubernetes.io/projected/e13d5469-5380-4b13-97dc-9a5ff72adcd6-kube-api-access-s67vq\") pod \"maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.672570 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.672539    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e13d5469-5380-4b13-97dc-9a5ff72adcd6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.773325 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.773301    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s67vq\" (UniqueName: \"kubernetes.io/projected/e13d5469-5380-4b13-97dc-9a5ff72adcd6-kube-api-access-s67vq\") pod \"maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.773471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.773334    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e13d5469-5380-4b13-97dc-9a5ff72adcd6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.773471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.773368    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13d5469-5380-4b13-97dc-9a5ff72adcd6-maas-api-tls\") pod \"maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.773991 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.773973    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e13d5469-5380-4b13-97dc-9a5ff72adcd6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.775658 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.775639    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13d5469-5380-4b13-97dc-9a5ff72adcd6-maas-api-tls\") pod \"maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.782155 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.782099    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s67vq\" (UniqueName: \"kubernetes.io/projected/e13d5469-5380-4b13-97dc-9a5ff72adcd6-kube-api-access-s67vq\") pod \"maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.844452 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.844436    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:57.972739 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:57.972715    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"]
Jul 26 16:15:57.975380 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:15:57.975351    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13d5469_5380_4b13_97dc_9a5ff72adcd6.slice/crio-107705b82757d813dd76562847331d3f11f29105085f7f1afbb09f35fc40819b WatchSource:0}: Error finding container 107705b82757d813dd76562847331d3f11f29105085f7f1afbb09f35fc40819b: Status 404 returned error can't find the container with id 107705b82757d813dd76562847331d3f11f29105085f7f1afbb09f35fc40819b
Jul 26 16:15:58.334974 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.334940    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"]
Jul 26 16:15:58.341993 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.341176    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:15:58.344065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.344041    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-aadfb2-dockercfg-fndtg\""
Jul 26 16:15:58.344184 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.344042    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-aadfb2\""
Jul 26 16:15:58.349067 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.349040    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"]
Jul 26 16:15:58.372015 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.371989    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"]
Jul 26 16:15:58.375878 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.375861    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:15:58.385958 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.385936    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"]
Jul 26 16:15:58.478542 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.478512    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kndz\" (UniqueName: \"kubernetes.io/projected/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-kube-api-access-6kndz\") pod \"payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266\" (UID: \"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:15:58.478652 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.478561    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdxkg\" (UniqueName: \"kubernetes.io/projected/cbfd5177-b27f-4771-8c24-80e86645aa39-kube-api-access-hdxkg\") pod \"payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj\" (UID: \"cbfd5177-b27f-4771-8c24-80e86645aa39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:15:58.478652 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.478603    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266\" (UID: \"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:15:58.478652 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.478630    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbfd5177-b27f-4771-8c24-80e86645aa39-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj\" (UID: \"cbfd5177-b27f-4771-8c24-80e86645aa39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:15:58.579050 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.579022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kndz\" (UniqueName: \"kubernetes.io/projected/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-kube-api-access-6kndz\") pod \"payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266\" (UID: \"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:15:58.579457 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.579061    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdxkg\" (UniqueName: \"kubernetes.io/projected/cbfd5177-b27f-4771-8c24-80e86645aa39-kube-api-access-hdxkg\") pod \"payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj\" (UID: \"cbfd5177-b27f-4771-8c24-80e86645aa39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:15:58.579457 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.579109    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266\" (UID: \"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:15:58.579457 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.579237    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbfd5177-b27f-4771-8c24-80e86645aa39-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj\" (UID: \"cbfd5177-b27f-4771-8c24-80e86645aa39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:15:58.579777 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.579759    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266\" (UID: \"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:15:58.580073 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.580055    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbfd5177-b27f-4771-8c24-80e86645aa39-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj\" (UID: \"cbfd5177-b27f-4771-8c24-80e86645aa39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:15:58.586863 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.586812    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdxkg\" (UniqueName: \"kubernetes.io/projected/cbfd5177-b27f-4771-8c24-80e86645aa39-kube-api-access-hdxkg\") pod \"payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj\" (UID: \"cbfd5177-b27f-4771-8c24-80e86645aa39\") " pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:15:58.586863 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.586816    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kndz\" (UniqueName: \"kubernetes.io/projected/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-kube-api-access-6kndz\") pod \"payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266\" (UID: \"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:15:58.653747 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.653722    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:15:58.684867 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.684821    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:15:58.803166 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.802768    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"]
Jul 26 16:15:58.806938 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:15:58.806299    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66d2afd7_6db6_4fdb_91f9_5735a6b9f0c8.slice/crio-9df3e0a7dfda9f67684fdf301178bccfe866cca6c500792c9f561105e44df0fb WatchSource:0}: Error finding container 9df3e0a7dfda9f67684fdf301178bccfe866cca6c500792c9f561105e44df0fb: Status 404 returned error can't find the container with id 9df3e0a7dfda9f67684fdf301178bccfe866cca6c500792c9f561105e44df0fb
Jul 26 16:15:58.828734 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.828711    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"]
Jul 26 16:15:58.831077 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:15:58.831045    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbfd5177_b27f_4771_8c24_80e86645aa39.slice/crio-b5d2b6d9764b445d3ab85e48e205e307030ac17fda16af69e0894bf4c654ba9d WatchSource:0}: Error finding container b5d2b6d9764b445d3ab85e48e205e307030ac17fda16af69e0894bf4c654ba9d: Status 404 returned error can't find the container with id b5d2b6d9764b445d3ab85e48e205e307030ac17fda16af69e0894bf4c654ba9d
Jul 26 16:15:58.870306 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.870278    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj" event={"ID":"cbfd5177-b27f-4771-8c24-80e86645aa39","Type":"ContainerStarted","Data":"b5d2b6d9764b445d3ab85e48e205e307030ac17fda16af69e0894bf4c654ba9d"}
Jul 26 16:15:58.871436 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.871414    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266" event={"ID":"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8","Type":"ContainerStarted","Data":"9df3e0a7dfda9f67684fdf301178bccfe866cca6c500792c9f561105e44df0fb"}
Jul 26 16:15:58.873090 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.873065    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x" event={"ID":"e13d5469-5380-4b13-97dc-9a5ff72adcd6","Type":"ContainerStarted","Data":"2afeb590f7ca54fe2b0e9c8cbe508547bda62e103c97ce0d52f972ace44da61b"}
Jul 26 16:15:58.873193 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.873123    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x" event={"ID":"e13d5469-5380-4b13-97dc-9a5ff72adcd6","Type":"ContainerStarted","Data":"107705b82757d813dd76562847331d3f11f29105085f7f1afbb09f35fc40819b"}
Jul 26 16:15:58.873193 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.873175    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:15:58.891341 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:58.891291    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x" podStartSLOduration=1.469461581 podStartE2EDuration="1.891272416s" totalImagesPullingTime="421.810835ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:15:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:15:58.889564193 +0000 UTC m=+2274.388954031" watchObservedRunningTime="2026-07-26 16:15:58.891272416 +0000 UTC m=+2274.390662257"
Jul 26 16:15:59.879132 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:59.879096    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj" event={"ID":"cbfd5177-b27f-4771-8c24-80e86645aa39","Type":"ContainerStarted","Data":"06d4c988045456793cde791c2fae111c7cc58e8d704eb10394c3b75eae3d5828"}
Jul 26 16:15:59.879594 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:59.879161    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:15:59.880639 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:59.880614    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266" event={"ID":"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8","Type":"ContainerStarted","Data":"6b00e710129e67858346f2c61fde785cec39ea5ad2057583d9d7c978b225559d"}
Jul 26 16:15:59.880811 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:59.880792    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:15:59.896215 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:59.896176    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj" podStartSLOduration=1.896162206 podStartE2EDuration="1.896162206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:15:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:15:59.894194848 +0000 UTC m=+2275.393584686" watchObservedRunningTime="2026-07-26 16:15:59.896162206 +0000 UTC m=+2275.395552085"
Jul 26 16:15:59.911079 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:15:59.911032    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266" podStartSLOduration=1.911016092 podStartE2EDuration="1.911016092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:15:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:15:59.909123813 +0000 UTC m=+2275.408513651" watchObservedRunningTime="2026-07-26 16:15:59.911016092 +0000 UTC m=+2275.410405930"
Jul 26 16:16:04.886679 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:04.886649    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:16:05.279277 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.279242    2560 scope.go:117] "RemoveContainer" containerID="48bfebd468a14c1f9921d228d32e6283d241996ae2eee22113d0e3dc1186c678"
Jul 26 16:16:05.539998 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.539882    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"]
Jul 26 16:16:05.545677 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.545655    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.547851 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.547828    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-e0208c-serving-cert\""
Jul 26 16:16:05.553806 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.553774    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"]
Jul 26 16:16:05.626765 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.626733    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzfxm\" (UniqueName: \"kubernetes.io/projected/05369022-95a6-403e-a1e2-0811151da7ba-kube-api-access-qzfxm\") pod \"maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.627003 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.626786    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05369022-95a6-403e-a1e2-0811151da7ba-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.627003 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.626813    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05369022-95a6-403e-a1e2-0811151da7ba-maas-api-tls\") pod \"maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.728099 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.728067    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzfxm\" (UniqueName: \"kubernetes.io/projected/05369022-95a6-403e-a1e2-0811151da7ba-kube-api-access-qzfxm\") pod \"maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.728289 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.728122    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05369022-95a6-403e-a1e2-0811151da7ba-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.728289 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.728151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05369022-95a6-403e-a1e2-0811151da7ba-maas-api-tls\") pod \"maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.728855 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.728828    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05369022-95a6-403e-a1e2-0811151da7ba-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.731810 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.731787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05369022-95a6-403e-a1e2-0811151da7ba-maas-api-tls\") pod \"maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.736610 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.736589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzfxm\" (UniqueName: \"kubernetes.io/projected/05369022-95a6-403e-a1e2-0811151da7ba-kube-api-access-qzfxm\") pod \"maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.858932 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.858817    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:05.995339 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:05.995312    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"]
Jul 26 16:16:05.997467 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:16:05.997440    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05369022_95a6_403e_a1e2_0811151da7ba.slice/crio-9e674e887db066f80e313cb3a490b402af582dacb7955ed8ea8f3d8315c12a64 WatchSource:0}: Error finding container 9e674e887db066f80e313cb3a490b402af582dacb7955ed8ea8f3d8315c12a64: Status 404 returned error can't find the container with id 9e674e887db066f80e313cb3a490b402af582dacb7955ed8ea8f3d8315c12a64
Jul 26 16:16:06.242133 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.242097    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"]
Jul 26 16:16:06.246644 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.246612    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:16:06.250974 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.250953    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-e0208c-dockercfg-kwvlp\""
Jul 26 16:16:06.251107 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.251090    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-e0208c\""
Jul 26 16:16:06.263373 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.263346    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"]
Jul 26 16:16:06.282279 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.282255    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"]
Jul 26 16:16:06.285514 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.285499    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:16:06.295667 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.295641    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"]
Jul 26 16:16:06.333943 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.333908    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4f039fb-3af1-4197-8f87-f564f47f771a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n\" (UID: \"a4f039fb-3af1-4197-8f87-f564f47f771a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:16:06.334105 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.333974    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9\" (UID: \"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:16:06.334105 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.333999    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkb6k\" (UniqueName: \"kubernetes.io/projected/a4f039fb-3af1-4197-8f87-f564f47f771a-kube-api-access-rkb6k\") pod \"payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n\" (UID: \"a4f039fb-3af1-4197-8f87-f564f47f771a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:16:06.334105 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.334048    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6262w\" (UniqueName: \"kubernetes.io/projected/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-kube-api-access-6262w\") pod \"payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9\" (UID: \"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:16:06.434507 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.434463    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9\" (UID: \"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:16:06.434507 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.434512    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkb6k\" (UniqueName: \"kubernetes.io/projected/a4f039fb-3af1-4197-8f87-f564f47f771a-kube-api-access-rkb6k\") pod \"payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n\" (UID: \"a4f039fb-3af1-4197-8f87-f564f47f771a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:16:06.434768 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.434539    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6262w\" (UniqueName: \"kubernetes.io/projected/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-kube-api-access-6262w\") pod \"payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9\" (UID: \"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:16:06.434768 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.434599    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4f039fb-3af1-4197-8f87-f564f47f771a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n\" (UID: \"a4f039fb-3af1-4197-8f87-f564f47f771a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:16:06.435258 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.435236    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9\" (UID: \"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:16:06.435361 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.435341    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4f039fb-3af1-4197-8f87-f564f47f771a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n\" (UID: \"a4f039fb-3af1-4197-8f87-f564f47f771a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:16:06.442606 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.442581    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6262w\" (UniqueName: \"kubernetes.io/projected/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-kube-api-access-6262w\") pod \"payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9\" (UID: \"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:16:06.442717 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.442633    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkb6k\" (UniqueName: \"kubernetes.io/projected/a4f039fb-3af1-4197-8f87-f564f47f771a-kube-api-access-rkb6k\") pod \"payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n\" (UID: \"a4f039fb-3af1-4197-8f87-f564f47f771a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:16:06.557146 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.557117    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:16:06.595913 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.595866    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:16:06.704045 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.704001    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"]
Jul 26 16:16:06.706283 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:16:06.706260    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4949ccdd_82f7_4a9f_88a7_fb9d96ec5382.slice/crio-fe51f1efaf20b9a789b301b39bbb03d8c8e9c5e6b59349d9ce333a747f7fb510 WatchSource:0}: Error finding container fe51f1efaf20b9a789b301b39bbb03d8c8e9c5e6b59349d9ce333a747f7fb510: Status 404 returned error can't find the container with id fe51f1efaf20b9a789b301b39bbb03d8c8e9c5e6b59349d9ce333a747f7fb510
Jul 26 16:16:06.755660 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.755631    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"]
Jul 26 16:16:06.759053 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:16:06.759020    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4f039fb_3af1_4197_8f87_f564f47f771a.slice/crio-acd368987459ea4ccf6c8c870873fe361950273c00af6cd9fc9ad6470bd09211 WatchSource:0}: Error finding container acd368987459ea4ccf6c8c870873fe361950273c00af6cd9fc9ad6470bd09211: Status 404 returned error can't find the container with id acd368987459ea4ccf6c8c870873fe361950273c00af6cd9fc9ad6470bd09211
Jul 26 16:16:06.920436 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.920396    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n" event={"ID":"a4f039fb-3af1-4197-8f87-f564f47f771a","Type":"ContainerStarted","Data":"b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2"}
Jul 26 16:16:06.920638 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.920463    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n" event={"ID":"a4f039fb-3af1-4197-8f87-f564f47f771a","Type":"ContainerStarted","Data":"acd368987459ea4ccf6c8c870873fe361950273c00af6cd9fc9ad6470bd09211"}
Jul 26 16:16:06.920936 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.920906    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:16:06.922785 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.922269    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs" event={"ID":"05369022-95a6-403e-a1e2-0811151da7ba","Type":"ContainerStarted","Data":"6266c434c20be5ad8d6300525fc499f92bc87192e7b28b8303250bb39ef028a0"}
Jul 26 16:16:06.922785 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.922319    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs" event={"ID":"05369022-95a6-403e-a1e2-0811151da7ba","Type":"ContainerStarted","Data":"9e674e887db066f80e313cb3a490b402af582dacb7955ed8ea8f3d8315c12a64"}
Jul 26 16:16:06.922785 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.922349    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:06.924039 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.924014    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9" event={"ID":"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382","Type":"ContainerStarted","Data":"9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e"}
Jul 26 16:16:06.924142 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.924061    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9" event={"ID":"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382","Type":"ContainerStarted","Data":"fe51f1efaf20b9a789b301b39bbb03d8c8e9c5e6b59349d9ce333a747f7fb510"}
Jul 26 16:16:06.924142 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.924100    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:16:06.935630 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.935573    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n" podStartSLOduration=0.935545904 podStartE2EDuration="935.545904ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:16:06.93486712 +0000 UTC m=+2282.434256957" watchObservedRunningTime="2026-07-26 16:16:06.935545904 +0000 UTC m=+2282.434935745"
Jul 26 16:16:06.949749 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.949692    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9" podStartSLOduration=0.949674118 podStartE2EDuration="949.674118ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:16:06.94924652 +0000 UTC m=+2282.448636413" watchObservedRunningTime="2026-07-26 16:16:06.949674118 +0000 UTC m=+2282.449063987"
Jul 26 16:16:06.969287 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:06.969234    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs" podStartSLOduration=1.44311859 podStartE2EDuration="1.969218564s" totalImagesPullingTime="526.099974ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:16:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:16:06.967677927 +0000 UTC m=+2282.467067761" watchObservedRunningTime="2026-07-26 16:16:06.969218564 +0000 UTC m=+2282.468608402"
Jul 26 16:16:10.888743 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:10.888715    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:16:10.889273 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:10.888830    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:16:12.935364 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:12.935324    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:16:17.931254 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:17.931224    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:16:17.931711 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:16:17.931485    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:20:05.338404 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:20:05.338374    2560 scope.go:117] "RemoveContainer" containerID="a8b8b69da70c0b453e146af12674042cecad08ed305705f85879a11091237507"
Jul 26 16:21:05.356455 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:21:05.356422    2560 scope.go:117] "RemoveContainer" containerID="9a9a72369deb2fb4892ab97714e7d5d6cbae8bffbf23a1ca2f730a2027bb489e"
Jul 26 16:21:05.364758 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:21:05.364738    2560 scope.go:117] "RemoveContainer" containerID="ec425ddb18e354428cdce726226a41707583b76d1932647488ff7bf16cb5c2f8"
Jul 26 16:21:05.371982 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:21:05.371960    2560 scope.go:117] "RemoveContainer" containerID="cfcdf21f9bef5405c0eae6680fc1671a4590fdcb0bc36962ddcdf408ecfa0460"
Jul 26 16:25:00.732834 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.732749    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"]
Jul 26 16:25:00.736453 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.736430    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:00.738570 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.738545    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-10e721-serving-cert\""
Jul 26 16:25:00.746108 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.746087    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"]
Jul 26 16:25:00.756079 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.756058    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59gdz\" (UniqueName: \"kubernetes.io/projected/170549ed-2881-4740-bb8c-4b6ec7b0373f-kube-api-access-59gdz\") pod \"maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:00.756203 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.756099    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/170549ed-2881-4740-bb8c-4b6ec7b0373f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:00.756203 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.756172    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170549ed-2881-4740-bb8c-4b6ec7b0373f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:00.856989 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.856959    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170549ed-2881-4740-bb8c-4b6ec7b0373f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:00.857123 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.857029    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59gdz\" (UniqueName: \"kubernetes.io/projected/170549ed-2881-4740-bb8c-4b6ec7b0373f-kube-api-access-59gdz\") pod \"maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:00.857123 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.857068    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/170549ed-2881-4740-bb8c-4b6ec7b0373f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:00.857211 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:25:00.857129    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-serving-cert: secret "maas-api-e2e-ipp-a-10e721-serving-cert" not found
Jul 26 16:25:00.857211 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:25:00.857200    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/170549ed-2881-4740-bb8c-4b6ec7b0373f-maas-api-tls podName:170549ed-2881-4740-bb8c-4b6ec7b0373f nodeName:}" failed. No retries permitted until 2026-07-26 16:25:01.357180172 +0000 UTC m=+2816.856569995 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/170549ed-2881-4740-bb8c-4b6ec7b0373f-maas-api-tls") pod "maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4" (UID: "170549ed-2881-4740-bb8c-4b6ec7b0373f") : secret "maas-api-e2e-ipp-a-10e721-serving-cert" not found
Jul 26 16:25:00.857711 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.857692    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/170549ed-2881-4740-bb8c-4b6ec7b0373f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:00.867912 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:00.867867    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59gdz\" (UniqueName: \"kubernetes.io/projected/170549ed-2881-4740-bb8c-4b6ec7b0373f-kube-api-access-59gdz\") pod \"maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:01.360851 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.360814    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170549ed-2881-4740-bb8c-4b6ec7b0373f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:01.363454 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.363412    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170549ed-2881-4740-bb8c-4b6ec7b0373f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:01.515224 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.515193    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"]
Jul 26 16:25:01.518697 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.518677    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:25:01.521029 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.521005    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-10e721-dockercfg-7fxq4\""
Jul 26 16:25:01.521223 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.521206    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-10e721\""
Jul 26 16:25:01.535969 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.535949    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"]
Jul 26 16:25:01.553793 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.553771    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"]
Jul 26 16:25:01.557038 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.557020    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:25:01.562512 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.562487    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cc2790d-a727-4593-83e1-f94511c4eeb9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2\" (UID: \"8cc2790d-a727-4593-83e1-f94511c4eeb9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:25:01.562624 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.562554    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cshs\" (UniqueName: \"kubernetes.io/projected/8cc2790d-a727-4593-83e1-f94511c4eeb9-kube-api-access-4cshs\") pod \"payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2\" (UID: \"8cc2790d-a727-4593-83e1-f94511c4eeb9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:25:01.568065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.568045    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"]
Jul 26 16:25:01.647917 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.647861    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:01.663246 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.663218    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cc2790d-a727-4593-83e1-f94511c4eeb9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2\" (UID: \"8cc2790d-a727-4593-83e1-f94511c4eeb9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:25:01.663339 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.663280    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d9ecf0-c007-43a1-9a89-252159f035a9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt\" (UID: \"a4d9ecf0-c007-43a1-9a89-252159f035a9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:25:01.663339 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.663307    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz7k8\" (UniqueName: \"kubernetes.io/projected/a4d9ecf0-c007-43a1-9a89-252159f035a9-kube-api-access-kz7k8\") pod \"payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt\" (UID: \"a4d9ecf0-c007-43a1-9a89-252159f035a9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:25:01.663449 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.663351    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cshs\" (UniqueName: \"kubernetes.io/projected/8cc2790d-a727-4593-83e1-f94511c4eeb9-kube-api-access-4cshs\") pod \"payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2\" (UID: \"8cc2790d-a727-4593-83e1-f94511c4eeb9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:25:01.664266 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.664244    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cc2790d-a727-4593-83e1-f94511c4eeb9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2\" (UID: \"8cc2790d-a727-4593-83e1-f94511c4eeb9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:25:01.671657 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.671634    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cshs\" (UniqueName: \"kubernetes.io/projected/8cc2790d-a727-4593-83e1-f94511c4eeb9-kube-api-access-4cshs\") pod \"payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2\" (UID: \"8cc2790d-a727-4593-83e1-f94511c4eeb9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:25:01.764649 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.764613    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d9ecf0-c007-43a1-9a89-252159f035a9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt\" (UID: \"a4d9ecf0-c007-43a1-9a89-252159f035a9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:25:01.764649 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.764660    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz7k8\" (UniqueName: \"kubernetes.io/projected/a4d9ecf0-c007-43a1-9a89-252159f035a9-kube-api-access-kz7k8\") pod \"payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt\" (UID: \"a4d9ecf0-c007-43a1-9a89-252159f035a9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:25:01.765509 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.765487    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d9ecf0-c007-43a1-9a89-252159f035a9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt\" (UID: \"a4d9ecf0-c007-43a1-9a89-252159f035a9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:25:01.772560 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.772532    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz7k8\" (UniqueName: \"kubernetes.io/projected/a4d9ecf0-c007-43a1-9a89-252159f035a9-kube-api-access-kz7k8\") pod \"payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt\" (UID: \"a4d9ecf0-c007-43a1-9a89-252159f035a9\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:25:01.789201 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.789048    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"]
Jul 26 16:25:01.791442 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:25:01.791410    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170549ed_2881_4740_bb8c_4b6ec7b0373f.slice/crio-fa1b0a1f8e5eeb37bf194a3e2232e19bc2b04f7e524480175ec131613d51b25d WatchSource:0}: Error finding container fa1b0a1f8e5eeb37bf194a3e2232e19bc2b04f7e524480175ec131613d51b25d: Status 404 returned error can't find the container with id fa1b0a1f8e5eeb37bf194a3e2232e19bc2b04f7e524480175ec131613d51b25d
Jul 26 16:25:01.792856 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.792840    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:25:01.829599 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.829572    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:25:01.868170 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.868140    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:25:01.978996 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:01.978945    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"]
Jul 26 16:25:01.982296 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:25:01.982267    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc2790d_a727_4593_83e1_f94511c4eeb9.slice/crio-04d9f32e2a03d1983b9bcae35eb35306d56f84673884522a9da7eded7eb688b2 WatchSource:0}: Error finding container 04d9f32e2a03d1983b9bcae35eb35306d56f84673884522a9da7eded7eb688b2: Status 404 returned error can't find the container with id 04d9f32e2a03d1983b9bcae35eb35306d56f84673884522a9da7eded7eb688b2
Jul 26 16:25:02.229576 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.229552    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"]
Jul 26 16:25:02.231853 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:25:02.231822    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d9ecf0_c007_43a1_9a89_252159f035a9.slice/crio-724e8891a4faf3e87c139db40671c93dddea30a6ae4626ca573c1a45cb25c920 WatchSource:0}: Error finding container 724e8891a4faf3e87c139db40671c93dddea30a6ae4626ca573c1a45cb25c920: Status 404 returned error can't find the container with id 724e8891a4faf3e87c139db40671c93dddea30a6ae4626ca573c1a45cb25c920
Jul 26 16:25:02.374720 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.374683    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4" event={"ID":"170549ed-2881-4740-bb8c-4b6ec7b0373f","Type":"ContainerStarted","Data":"f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72"}
Jul 26 16:25:02.374933 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.374851    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:02.374933 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.374910    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4" event={"ID":"170549ed-2881-4740-bb8c-4b6ec7b0373f","Type":"ContainerStarted","Data":"fa1b0a1f8e5eeb37bf194a3e2232e19bc2b04f7e524480175ec131613d51b25d"}
Jul 26 16:25:02.376295 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.376269    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2" event={"ID":"8cc2790d-a727-4593-83e1-f94511c4eeb9","Type":"ContainerStarted","Data":"827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801"}
Jul 26 16:25:02.376425 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.376312    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:25:02.376425 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.376346    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2" event={"ID":"8cc2790d-a727-4593-83e1-f94511c4eeb9","Type":"ContainerStarted","Data":"04d9f32e2a03d1983b9bcae35eb35306d56f84673884522a9da7eded7eb688b2"}
Jul 26 16:25:02.377795 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.377772    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt" event={"ID":"a4d9ecf0-c007-43a1-9a89-252159f035a9","Type":"ContainerStarted","Data":"4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1"}
Jul 26 16:25:02.377882 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.377816    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt" event={"ID":"a4d9ecf0-c007-43a1-9a89-252159f035a9","Type":"ContainerStarted","Data":"724e8891a4faf3e87c139db40671c93dddea30a6ae4626ca573c1a45cb25c920"}
Jul 26 16:25:02.377882 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.377847    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:25:02.393649 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.393608    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4" podStartSLOduration=1.945973477 podStartE2EDuration="2.393593572s" totalImagesPullingTime="447.620095ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:25:02.391718355 +0000 UTC m=+2817.891108195" watchObservedRunningTime="2026-07-26 16:25:02.393593572 +0000 UTC m=+2817.892983434"
Jul 26 16:25:02.407267 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.407224    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2" podStartSLOduration=1.407208628 podStartE2EDuration="1.407208628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:25:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:25:02.406671399 +0000 UTC m=+2817.906061236" watchObservedRunningTime="2026-07-26 16:25:02.407208628 +0000 UTC m=+2817.906598469"
Jul 26 16:25:02.424728 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:02.424681    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt" podStartSLOduration=1.424669261 podStartE2EDuration="1.424669261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:25:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:25:02.422403746 +0000 UTC m=+2817.921793588" watchObservedRunningTime="2026-07-26 16:25:02.424669261 +0000 UTC m=+2817.924059113"
Jul 26 16:25:08.387815 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:08.387790    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:25:13.384775 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:13.384744    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:25:13.385186 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:13.384820    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:25:22.220320 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.220289    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"]
Jul 26 16:25:22.223712 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.223689    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.226135 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.226110    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-8c3dba-serving-cert\""
Jul 26 16:25:22.234641 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.234068    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"]
Jul 26 16:25:22.318484 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.318457    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6wkv\" (UniqueName: \"kubernetes.io/projected/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-kube-api-access-v6wkv\") pod \"maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.318625 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.318524    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.318625 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.318597    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.419694 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.419660    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.419841 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.419707    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.419841 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.419745    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wkv\" (UniqueName: \"kubernetes.io/projected/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-kube-api-access-v6wkv\") pod \"maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.420430 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.420406    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.422145 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.422125    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.427283 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.427264    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6wkv\" (UniqueName: \"kubernetes.io/projected/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-kube-api-access-v6wkv\") pod \"maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.540356 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.540284    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:22.673587 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.673558    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"]
Jul 26 16:25:22.675224 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:25:22.675191    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd9043c_0b68_428d_b2a6_f5bed707a7a0.slice/crio-fa7857d65dc7948c82240f51ce248280e8eabcee86c25c533a8305d59cca3e3e WatchSource:0}: Error finding container fa7857d65dc7948c82240f51ce248280e8eabcee86c25c533a8305d59cca3e3e: Status 404 returned error can't find the container with id fa7857d65dc7948c82240f51ce248280e8eabcee86c25c533a8305d59cca3e3e
Jul 26 16:25:22.966113 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.966085    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"]
Jul 26 16:25:22.970576 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.970556    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:25:22.972860 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.972840    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-8c3dba\""
Jul 26 16:25:22.973681 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.973660    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-8c3dba-dockercfg-bh78s\""
Jul 26 16:25:22.981902 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:22.981864    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"]
Jul 26 16:25:23.025877 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.025851    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4441b18-67b8-4fbe-908e-5601948312d2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp\" (UID: \"d4441b18-67b8-4fbe-908e-5601948312d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:25:23.026002 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.025975    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzckm\" (UniqueName: \"kubernetes.io/projected/d4441b18-67b8-4fbe-908e-5601948312d2-kube-api-access-jzckm\") pod \"payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp\" (UID: \"d4441b18-67b8-4fbe-908e-5601948312d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:25:23.126471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.126444    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4441b18-67b8-4fbe-908e-5601948312d2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp\" (UID: \"d4441b18-67b8-4fbe-908e-5601948312d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:25:23.126587 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.126548    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzckm\" (UniqueName: \"kubernetes.io/projected/d4441b18-67b8-4fbe-908e-5601948312d2-kube-api-access-jzckm\") pod \"payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp\" (UID: \"d4441b18-67b8-4fbe-908e-5601948312d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:25:23.127210 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.127126    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4441b18-67b8-4fbe-908e-5601948312d2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp\" (UID: \"d4441b18-67b8-4fbe-908e-5601948312d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:25:23.134557 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.134527    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzckm\" (UniqueName: \"kubernetes.io/projected/d4441b18-67b8-4fbe-908e-5601948312d2-kube-api-access-jzckm\") pod \"payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp\" (UID: \"d4441b18-67b8-4fbe-908e-5601948312d2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:25:23.282295 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.282223    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:25:23.472223 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.472187    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9" event={"ID":"6bd9043c-0b68-428d-b2a6-f5bed707a7a0","Type":"ContainerStarted","Data":"a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d"}
Jul 26 16:25:23.472364 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.472244    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:23.472364 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.472261    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9" event={"ID":"6bd9043c-0b68-428d-b2a6-f5bed707a7a0","Type":"ContainerStarted","Data":"fa7857d65dc7948c82240f51ce248280e8eabcee86c25c533a8305d59cca3e3e"}
Jul 26 16:25:23.488425 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.488379    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9" podStartSLOduration=1.084718021 podStartE2EDuration="1.488364s" totalImagesPullingTime="403.645979ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:25:23.48718705 +0000 UTC m=+2838.986576911" watchObservedRunningTime="2026-07-26 16:25:23.488364 +0000 UTC m=+2838.987753837"
Jul 26 16:25:23.622373 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:23.622302    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"]
Jul 26 16:25:23.625214 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:25:23.625187    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4441b18_67b8_4fbe_908e_5601948312d2.slice/crio-591d2702be7c144616c85f9c98032f2c5980041e1626406789cb33e3a6b0b6db WatchSource:0}: Error finding container 591d2702be7c144616c85f9c98032f2c5980041e1626406789cb33e3a6b0b6db: Status 404 returned error can't find the container with id 591d2702be7c144616c85f9c98032f2c5980041e1626406789cb33e3a6b0b6db
Jul 26 16:25:24.479154 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:24.479121    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp" event={"ID":"d4441b18-67b8-4fbe-908e-5601948312d2","Type":"ContainerStarted","Data":"88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d"}
Jul 26 16:25:24.479603 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:24.479174    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp" event={"ID":"d4441b18-67b8-4fbe-908e-5601948312d2","Type":"ContainerStarted","Data":"591d2702be7c144616c85f9c98032f2c5980041e1626406789cb33e3a6b0b6db"}
Jul 26 16:25:24.479603 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:24.479377    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:25:24.495659 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:24.495616    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp" podStartSLOduration=2.495603141 podStartE2EDuration="2.495603141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:25:24.493307889 +0000 UTC m=+2839.992697727" watchObservedRunningTime="2026-07-26 16:25:24.495603141 +0000 UTC m=+2839.994992978"
Jul 26 16:25:29.484992 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:29.484965    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:25:35.486589 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:35.486545    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:25:48.562019 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:48.561984    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-pnh69_f1a3706c-d0ff-4751-8639-a1de8788e821/payload-processing/0.log"
Jul 26 16:25:48.673960 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:25:48.673934    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp_d4441b18-67b8-4fbe-908e-5601948312d2/payload-processing/0.log"
Jul 26 16:26:45.729108 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:45.729077    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt_a4d9ecf0-c007-43a1-9a89-252159f035a9/payload-processing/0.log"
Jul 26 16:26:45.848559 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:45.848530    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-pnh69_f1a3706c-d0ff-4751-8639-a1de8788e821/payload-processing/0.log"
Jul 26 16:26:45.970625 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:45.970597    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp_d4441b18-67b8-4fbe-908e-5601948312d2/payload-processing/0.log"
Jul 26 16:26:51.781456 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:51.781424    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"]
Jul 26 16:26:51.785010 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:51.784989    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:51.787165 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:51.787138    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-eb0742-serving-cert\""
Jul 26 16:26:51.795060 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:51.795014    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"]
Jul 26 16:26:51.973376 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:51.973335    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:51.973524 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:51.973399    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:51.973524 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:51.973454    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt24v\" (UniqueName: \"kubernetes.io/projected/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-kube-api-access-dt24v\") pod \"maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:52.073811 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:52.073744    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:52.073811 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:52.073784    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:52.073811 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:52.073807    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt24v\" (UniqueName: \"kubernetes.io/projected/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-kube-api-access-dt24v\") pod \"maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:52.074033 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:26:52.073928    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-serving-cert: secret "maas-api-e2e-ipp-cleanup-eb0742-serving-cert" not found
Jul 26 16:26:52.074033 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:26:52.074025    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-maas-api-tls podName:b55f70f2-a512-4639-922c-4ed9c2cd8ff6 nodeName:}" failed. No retries permitted until 2026-07-26 16:26:52.574002684 +0000 UTC m=+2928.073392501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj" (UID: "b55f70f2-a512-4639-922c-4ed9c2cd8ff6") : secret "maas-api-e2e-ipp-cleanup-eb0742-serving-cert" not found
Jul 26 16:26:52.074465 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:52.074446    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:52.084653 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:52.084634    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt24v\" (UniqueName: \"kubernetes.io/projected/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-kube-api-access-dt24v\") pod \"maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:52.578835 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:52.578796    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:52.581345 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:52.581314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:52.696586 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:52.696557    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:52.832446 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:52.832414    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"]
Jul 26 16:26:52.833773 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:26:52.833746    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55f70f2_a512_4639_922c_4ed9c2cd8ff6.slice/crio-daf8761e8218eaed4a99eb2dbb2daa95f6a4ded4f8e5c5c5c2b1e69892ae2427 WatchSource:0}: Error finding container daf8761e8218eaed4a99eb2dbb2daa95f6a4ded4f8e5c5c5c2b1e69892ae2427: Status 404 returned error can't find the container with id daf8761e8218eaed4a99eb2dbb2daa95f6a4ded4f8e5c5c5c2b1e69892ae2427
Jul 26 16:26:52.898977 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:52.898948    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj" event={"ID":"b55f70f2-a512-4639-922c-4ed9c2cd8ff6","Type":"ContainerStarted","Data":"daf8761e8218eaed4a99eb2dbb2daa95f6a4ded4f8e5c5c5c2b1e69892ae2427"}
Jul 26 16:26:53.905719 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:53.905678    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj" event={"ID":"b55f70f2-a512-4639-922c-4ed9c2cd8ff6","Type":"ContainerStarted","Data":"bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e"}
Jul 26 16:26:53.906170 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:53.905762    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:26:53.921624 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:53.921578    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj" podStartSLOduration=2.5188837 podStartE2EDuration="2.921564214s" totalImagesPullingTime="402.680514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:26:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:26:53.919251533 +0000 UTC m=+2929.418641371" watchObservedRunningTime="2026-07-26 16:26:53.921564214 +0000 UTC m=+2929.420954051"
Jul 26 16:26:59.915793 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:26:59.915761    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:27:09.737715 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:09.737681    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"]
Jul 26 16:27:09.742468 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:09.742447    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"
Jul 26 16:27:09.744313 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:09.744295    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:27:09.748762 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:09.748741    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"]
Jul 26 16:27:09.816871 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:09.816848    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38403683-79f4-424b-9503-fdeca74593b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs\" (UID: \"38403683-79f4-424b-9503-fdeca74593b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"
Jul 26 16:27:09.917612 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:09.917583    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38403683-79f4-424b-9503-fdeca74593b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs\" (UID: \"38403683-79f4-424b-9503-fdeca74593b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"
Jul 26 16:27:09.918185 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:09.918167    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38403683-79f4-424b-9503-fdeca74593b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs\" (UID: \"38403683-79f4-424b-9503-fdeca74593b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"
Jul 26 16:27:10.054726 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:10.054661    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"
Jul 26 16:27:10.190099 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:10.190073    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"]
Jul 26 16:27:10.192158 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:27:10.192129    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38403683_79f4_424b_9503_fdeca74593b4.slice/crio-c8c9945ee84878803a1c14170edea6042770d01b41270163283958b211dd071a WatchSource:0}: Error finding container c8c9945ee84878803a1c14170edea6042770d01b41270163283958b211dd071a: Status 404 returned error can't find the container with id c8c9945ee84878803a1c14170edea6042770d01b41270163283958b211dd071a
Jul 26 16:27:10.987845 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:10.987810    2560 generic.go:356] "Generic (PLEG): container finished" podID="38403683-79f4-424b-9503-fdeca74593b4" containerID="45f7bc32d2187738f53ec275208e5bfbfe1ee739ee05657e85c639973ca24a57" exitCode=0
Jul 26 16:27:10.988164 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:10.987908    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs" event={"ID":"38403683-79f4-424b-9503-fdeca74593b4","Type":"ContainerDied","Data":"45f7bc32d2187738f53ec275208e5bfbfe1ee739ee05657e85c639973ca24a57"}
Jul 26 16:27:10.988164 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:10.987961    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs" event={"ID":"38403683-79f4-424b-9503-fdeca74593b4","Type":"ContainerStarted","Data":"c8c9945ee84878803a1c14170edea6042770d01b41270163283958b211dd071a"}
Jul 26 16:27:12.131936 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:12.131909    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"
Jul 26 16:27:12.235819 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:12.235789    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38403683-79f4-424b-9503-fdeca74593b4-maas-api-service-ca\") pod \"38403683-79f4-424b-9503-fdeca74593b4\" (UID: \"38403683-79f4-424b-9503-fdeca74593b4\") "
Jul 26 16:27:12.236146 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:12.236123    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38403683-79f4-424b-9503-fdeca74593b4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38403683-79f4-424b-9503-fdeca74593b4" (UID: "38403683-79f4-424b-9503-fdeca74593b4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:27:12.337044 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:12.336970    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38403683-79f4-424b-9503-fdeca74593b4-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:12.999338 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:12.999308    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"
Jul 26 16:27:12.999496 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:12.999304    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs" event={"ID":"38403683-79f4-424b-9503-fdeca74593b4","Type":"ContainerDied","Data":"c8c9945ee84878803a1c14170edea6042770d01b41270163283958b211dd071a"}
Jul 26 16:27:12.999496 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:12.999433    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8c9945ee84878803a1c14170edea6042770d01b41270163283958b211dd071a"
Jul 26 16:27:21.375729 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.375695    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"]
Jul 26 16:27:21.376194 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.375901    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj" podUID="b55f70f2-a512-4639-922c-4ed9c2cd8ff6" containerName="maas-api" containerID="cri-o://bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e" gracePeriod=30
Jul 26 16:27:21.640796 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.640771    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:27:21.706640 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.706615    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt24v\" (UniqueName: \"kubernetes.io/projected/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-kube-api-access-dt24v\") pod \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") "
Jul 26 16:27:21.706792 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.706678    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-openshift-service-ca\") pod \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") "
Jul 26 16:27:21.706792 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.706698    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-maas-api-tls\") pod \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\" (UID: \"b55f70f2-a512-4639-922c-4ed9c2cd8ff6\") "
Jul 26 16:27:21.707040 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.707016    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b55f70f2-a512-4639-922c-4ed9c2cd8ff6" (UID: "b55f70f2-a512-4639-922c-4ed9c2cd8ff6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:27:21.709008 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.708982    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b55f70f2-a512-4639-922c-4ed9c2cd8ff6" (UID: "b55f70f2-a512-4639-922c-4ed9c2cd8ff6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:27:21.709301 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.709272    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-kube-api-access-dt24v" (OuterVolumeSpecName: "kube-api-access-dt24v") pod "b55f70f2-a512-4639-922c-4ed9c2cd8ff6" (UID: "b55f70f2-a512-4639-922c-4ed9c2cd8ff6"). InnerVolumeSpecName "kube-api-access-dt24v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:27:21.807848 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.807822    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt24v\" (UniqueName: \"kubernetes.io/projected/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-kube-api-access-dt24v\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:21.807848 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.807844    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:21.808006 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:21.807854    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b55f70f2-a512-4639-922c-4ed9c2cd8ff6-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:22.044157 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:22.044083    2560 generic.go:356] "Generic (PLEG): container finished" podID="b55f70f2-a512-4639-922c-4ed9c2cd8ff6" containerID="bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e" exitCode=0
Jul 26 16:27:22.044157 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:22.044127    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"
Jul 26 16:27:22.044314 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:22.044165    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj" event={"ID":"b55f70f2-a512-4639-922c-4ed9c2cd8ff6","Type":"ContainerDied","Data":"bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e"}
Jul 26 16:27:22.044314 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:22.044218    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj" event={"ID":"b55f70f2-a512-4639-922c-4ed9c2cd8ff6","Type":"ContainerDied","Data":"daf8761e8218eaed4a99eb2dbb2daa95f6a4ded4f8e5c5c5c2b1e69892ae2427"}
Jul 26 16:27:22.044314 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:22.044240    2560 scope.go:117] "RemoveContainer" containerID="bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e"
Jul 26 16:27:22.053742 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:22.053725    2560 scope.go:117] "RemoveContainer" containerID="bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e"
Jul 26 16:27:22.054003 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:27:22.053983    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e\": container with ID starting with bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e not found: ID does not exist" containerID="bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e"
Jul 26 16:27:22.054061 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:22.054013    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e"} err="failed to get container status \"bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e\": rpc error: code = NotFound desc = could not find container \"bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e\": container with ID starting with bea7efd92331a52cadf51a0d93c18e15c14f874d42bcc5257f58baec4135057e not found: ID does not exist"
Jul 26 16:27:22.064456 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:22.064436    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"]
Jul 26 16:27:22.067777 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:22.067757    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eb0742-566d689bdf-xzvdj"]
Jul 26 16:27:23.138682 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:23.138652    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b55f70f2-a512-4639-922c-4ed9c2cd8ff6" path="/var/lib/kubelet/pods/b55f70f2-a512-4639-922c-4ed9c2cd8ff6/volumes"
Jul 26 16:27:24.811804 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:24.811770    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"]
Jul 26 16:27:24.814938 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:24.814913    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-eb0742-g4hrs"]
Jul 26 16:27:25.138923 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:25.138873    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38403683-79f4-424b-9503-fdeca74593b4" path="/var/lib/kubelet/pods/38403683-79f4-424b-9503-fdeca74593b4/volumes"
Jul 26 16:27:53.433715 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.433635    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"]
Jul 26 16:27:53.434302 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.433869    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4" podUID="170549ed-2881-4740-bb8c-4b6ec7b0373f" containerName="maas-api" containerID="cri-o://f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72" gracePeriod=30
Jul 26 16:27:53.490455 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.490429    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"]
Jul 26 16:27:53.490667 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.490640    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt" podUID="a4d9ecf0-c007-43a1-9a89-252159f035a9" containerName="payload-processing" containerID="cri-o://4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1" gracePeriod=30
Jul 26 16:27:53.496312 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.496268    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"]
Jul 26 16:27:53.496648 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.496490    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2" podUID="8cc2790d-a727-4593-83e1-f94511c4eeb9" containerName="payload-pre-processing" containerID="cri-o://827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801" gracePeriod=30
Jul 26 16:27:53.781190 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.781166    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:27:53.807012 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.806991    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:27:53.810080 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.810057    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:27:53.859606 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.859581    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170549ed-2881-4740-bb8c-4b6ec7b0373f-maas-api-tls\") pod \"170549ed-2881-4740-bb8c-4b6ec7b0373f\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") "
Jul 26 16:27:53.859752 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.859694    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59gdz\" (UniqueName: \"kubernetes.io/projected/170549ed-2881-4740-bb8c-4b6ec7b0373f-kube-api-access-59gdz\") pod \"170549ed-2881-4740-bb8c-4b6ec7b0373f\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") "
Jul 26 16:27:53.859825 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.859768    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/170549ed-2881-4740-bb8c-4b6ec7b0373f-openshift-service-ca\") pod \"170549ed-2881-4740-bb8c-4b6ec7b0373f\" (UID: \"170549ed-2881-4740-bb8c-4b6ec7b0373f\") "
Jul 26 16:27:53.860139 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.860115    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/170549ed-2881-4740-bb8c-4b6ec7b0373f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "170549ed-2881-4740-bb8c-4b6ec7b0373f" (UID: "170549ed-2881-4740-bb8c-4b6ec7b0373f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:27:53.861516 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.861496    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/170549ed-2881-4740-bb8c-4b6ec7b0373f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "170549ed-2881-4740-bb8c-4b6ec7b0373f" (UID: "170549ed-2881-4740-bb8c-4b6ec7b0373f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:27:53.861615 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.861594    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/170549ed-2881-4740-bb8c-4b6ec7b0373f-kube-api-access-59gdz" (OuterVolumeSpecName: "kube-api-access-59gdz") pod "170549ed-2881-4740-bb8c-4b6ec7b0373f" (UID: "170549ed-2881-4740-bb8c-4b6ec7b0373f"). InnerVolumeSpecName "kube-api-access-59gdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:27:53.960873 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.960808    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz7k8\" (UniqueName: \"kubernetes.io/projected/a4d9ecf0-c007-43a1-9a89-252159f035a9-kube-api-access-kz7k8\") pod \"a4d9ecf0-c007-43a1-9a89-252159f035a9\" (UID: \"a4d9ecf0-c007-43a1-9a89-252159f035a9\") "
Jul 26 16:27:53.960873 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.960857    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cshs\" (UniqueName: \"kubernetes.io/projected/8cc2790d-a727-4593-83e1-f94511c4eeb9-kube-api-access-4cshs\") pod \"8cc2790d-a727-4593-83e1-f94511c4eeb9\" (UID: \"8cc2790d-a727-4593-83e1-f94511c4eeb9\") "
Jul 26 16:27:53.961066 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.960976    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d9ecf0-c007-43a1-9a89-252159f035a9-plugins-config-volume\") pod \"a4d9ecf0-c007-43a1-9a89-252159f035a9\" (UID: \"a4d9ecf0-c007-43a1-9a89-252159f035a9\") "
Jul 26 16:27:53.961066 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.961006    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cc2790d-a727-4593-83e1-f94511c4eeb9-plugins-config-volume\") pod \"8cc2790d-a727-4593-83e1-f94511c4eeb9\" (UID: \"8cc2790d-a727-4593-83e1-f94511c4eeb9\") "
Jul 26 16:27:53.961232 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.961217    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/170549ed-2881-4740-bb8c-4b6ec7b0373f-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:53.961292 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.961239    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59gdz\" (UniqueName: \"kubernetes.io/projected/170549ed-2881-4740-bb8c-4b6ec7b0373f-kube-api-access-59gdz\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:53.961292 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.961254    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/170549ed-2881-4740-bb8c-4b6ec7b0373f-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:53.961357 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.961328    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4d9ecf0-c007-43a1-9a89-252159f035a9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a4d9ecf0-c007-43a1-9a89-252159f035a9" (UID: "a4d9ecf0-c007-43a1-9a89-252159f035a9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:27:53.961396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.961374    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cc2790d-a727-4593-83e1-f94511c4eeb9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8cc2790d-a727-4593-83e1-f94511c4eeb9" (UID: "8cc2790d-a727-4593-83e1-f94511c4eeb9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:27:53.962666 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.962646    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4d9ecf0-c007-43a1-9a89-252159f035a9-kube-api-access-kz7k8" (OuterVolumeSpecName: "kube-api-access-kz7k8") pod "a4d9ecf0-c007-43a1-9a89-252159f035a9" (UID: "a4d9ecf0-c007-43a1-9a89-252159f035a9"). InnerVolumeSpecName "kube-api-access-kz7k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:27:53.962724 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:53.962691    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cc2790d-a727-4593-83e1-f94511c4eeb9-kube-api-access-4cshs" (OuterVolumeSpecName: "kube-api-access-4cshs") pod "8cc2790d-a727-4593-83e1-f94511c4eeb9" (UID: "8cc2790d-a727-4593-83e1-f94511c4eeb9"). InnerVolumeSpecName "kube-api-access-4cshs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:27:54.061827 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.061801    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d9ecf0-c007-43a1-9a89-252159f035a9-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:54.061827 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.061825    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cc2790d-a727-4593-83e1-f94511c4eeb9-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:54.061985 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.061836    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz7k8\" (UniqueName: \"kubernetes.io/projected/a4d9ecf0-c007-43a1-9a89-252159f035a9-kube-api-access-kz7k8\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:54.061985 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.061845    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cshs\" (UniqueName: \"kubernetes.io/projected/8cc2790d-a727-4593-83e1-f94511c4eeb9-kube-api-access-4cshs\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:27:54.192538 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.192508    2560 generic.go:356] "Generic (PLEG): container finished" podID="170549ed-2881-4740-bb8c-4b6ec7b0373f" containerID="f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72" exitCode=0
Jul 26 16:27:54.192632 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.192558    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"
Jul 26 16:27:54.192694 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.192555    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4" event={"ID":"170549ed-2881-4740-bb8c-4b6ec7b0373f","Type":"ContainerDied","Data":"f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72"}
Jul 26 16:27:54.192756 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.192678    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4" event={"ID":"170549ed-2881-4740-bb8c-4b6ec7b0373f","Type":"ContainerDied","Data":"fa1b0a1f8e5eeb37bf194a3e2232e19bc2b04f7e524480175ec131613d51b25d"}
Jul 26 16:27:54.192756 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.192720    2560 scope.go:117] "RemoveContainer" containerID="f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72"
Jul 26 16:27:54.194207 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.194188    2560 generic.go:356] "Generic (PLEG): container finished" podID="8cc2790d-a727-4593-83e1-f94511c4eeb9" containerID="827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801" exitCode=0
Jul 26 16:27:54.194306 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.194237    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"
Jul 26 16:27:54.194306 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.194255    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2" event={"ID":"8cc2790d-a727-4593-83e1-f94511c4eeb9","Type":"ContainerDied","Data":"827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801"}
Jul 26 16:27:54.194306 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.194301    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2" event={"ID":"8cc2790d-a727-4593-83e1-f94511c4eeb9","Type":"ContainerDied","Data":"04d9f32e2a03d1983b9bcae35eb35306d56f84673884522a9da7eded7eb688b2"}
Jul 26 16:27:54.195638 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.195618    2560 generic.go:356] "Generic (PLEG): container finished" podID="a4d9ecf0-c007-43a1-9a89-252159f035a9" containerID="4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1" exitCode=0
Jul 26 16:27:54.195752 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.195669    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"
Jul 26 16:27:54.195752 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.195670    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt" event={"ID":"a4d9ecf0-c007-43a1-9a89-252159f035a9","Type":"ContainerDied","Data":"4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1"}
Jul 26 16:27:54.195752 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.195718    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt" event={"ID":"a4d9ecf0-c007-43a1-9a89-252159f035a9","Type":"ContainerDied","Data":"724e8891a4faf3e87c139db40671c93dddea30a6ae4626ca573c1a45cb25c920"}
Jul 26 16:27:54.203978 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.203901    2560 scope.go:117] "RemoveContainer" containerID="f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72"
Jul 26 16:27:54.204206 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:27:54.204179    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72\": container with ID starting with f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72 not found: ID does not exist" containerID="f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72"
Jul 26 16:27:54.204287 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.204215    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72"} err="failed to get container status \"f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72\": rpc error: code = NotFound desc = could not find container \"f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72\": container with ID starting with f5f735689f1d557f7281d1e198adf240f9079ca35390298495695e0e09949f72 not found: ID does not exist"
Jul 26 16:27:54.204287 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.204239    2560 scope.go:117] "RemoveContainer" containerID="827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801"
Jul 26 16:27:54.213141 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.213126    2560 scope.go:117] "RemoveContainer" containerID="827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801"
Jul 26 16:27:54.213380 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:27:54.213365    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801\": container with ID starting with 827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801 not found: ID does not exist" containerID="827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801"
Jul 26 16:27:54.213425 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.213386    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801"} err="failed to get container status \"827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801\": rpc error: code = NotFound desc = could not find container \"827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801\": container with ID starting with 827994be99b1b584b95db3884b22926c584534b280befe3cf06cdcf05e95d801 not found: ID does not exist"
Jul 26 16:27:54.213425 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.213402    2560 scope.go:117] "RemoveContainer" containerID="4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1"
Jul 26 16:27:54.218192 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.218169    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"]
Jul 26 16:27:54.221386 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.221371    2560 scope.go:117] "RemoveContainer" containerID="4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1"
Jul 26 16:27:54.221592 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:27:54.221574    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1\": container with ID starting with 4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1 not found: ID does not exist" containerID="4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1"
Jul 26 16:27:54.221660 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.221601    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1"} err="failed to get container status \"4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1\": rpc error: code = NotFound desc = could not find container \"4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1\": container with ID starting with 4ae80bb573b525b654edfbdfe3dd152b92bc9baab394a759456c00b49089b9e1 not found: ID does not exist"
Jul 26 16:27:54.222713 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.222694    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10e721-68b47dcf44-8cbf4"]
Jul 26 16:27:54.231509 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.231488    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"]
Jul 26 16:27:54.234967 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.234947    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-10e721-5cbfc458d-t8zb2"]
Jul 26 16:27:54.244298 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.244281    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"]
Jul 26 16:27:54.255483 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:54.252465    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-10e721-59547d45d-dv5qt"]
Jul 26 16:27:55.140355 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:55.140322    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="170549ed-2881-4740-bb8c-4b6ec7b0373f" path="/var/lib/kubelet/pods/170549ed-2881-4740-bb8c-4b6ec7b0373f/volumes"
Jul 26 16:27:55.140711 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:55.140676    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cc2790d-a727-4593-83e1-f94511c4eeb9" path="/var/lib/kubelet/pods/8cc2790d-a727-4593-83e1-f94511c4eeb9/volumes"
Jul 26 16:27:55.140997 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:27:55.140984    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4d9ecf0-c007-43a1-9a89-252159f035a9" path="/var/lib/kubelet/pods/a4d9ecf0-c007-43a1-9a89-252159f035a9/volumes"
Jul 26 16:28:55.589791 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:55.589759    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"]
Jul 26 16:28:55.590375 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:55.590063    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9" podUID="6bd9043c-0b68-428d-b2a6-f5bed707a7a0" containerName="maas-api" containerID="cri-o://a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d" gracePeriod=30
Jul 26 16:28:55.647466 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:55.647438    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"]
Jul 26 16:28:55.647698 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:55.647671    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp" podUID="d4441b18-67b8-4fbe-908e-5601948312d2" containerName="payload-processing" containerID="cri-o://88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d" gracePeriod=30
Jul 26 16:28:55.912873 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:55.912848    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:28:55.944394 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:55.944374    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:28:56.055385 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.055361    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-maas-api-tls\") pod \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") "
Jul 26 16:28:56.055534 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.055396    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4441b18-67b8-4fbe-908e-5601948312d2-plugins-config-volume\") pod \"d4441b18-67b8-4fbe-908e-5601948312d2\" (UID: \"d4441b18-67b8-4fbe-908e-5601948312d2\") "
Jul 26 16:28:56.055534 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.055453    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6wkv\" (UniqueName: \"kubernetes.io/projected/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-kube-api-access-v6wkv\") pod \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") "
Jul 26 16:28:56.055534 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.055470    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzckm\" (UniqueName: \"kubernetes.io/projected/d4441b18-67b8-4fbe-908e-5601948312d2-kube-api-access-jzckm\") pod \"d4441b18-67b8-4fbe-908e-5601948312d2\" (UID: \"d4441b18-67b8-4fbe-908e-5601948312d2\") "
Jul 26 16:28:56.055534 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.055500    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-openshift-service-ca\") pod \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\" (UID: \"6bd9043c-0b68-428d-b2a6-f5bed707a7a0\") "
Jul 26 16:28:56.055948 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.055917    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4441b18-67b8-4fbe-908e-5601948312d2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4441b18-67b8-4fbe-908e-5601948312d2" (UID: "d4441b18-67b8-4fbe-908e-5601948312d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:28:56.056007 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.055944    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6bd9043c-0b68-428d-b2a6-f5bed707a7a0" (UID: "6bd9043c-0b68-428d-b2a6-f5bed707a7a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:28:56.057474 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.057451    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-kube-api-access-v6wkv" (OuterVolumeSpecName: "kube-api-access-v6wkv") pod "6bd9043c-0b68-428d-b2a6-f5bed707a7a0" (UID: "6bd9043c-0b68-428d-b2a6-f5bed707a7a0"). InnerVolumeSpecName "kube-api-access-v6wkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:28:56.057797 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.057779    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4441b18-67b8-4fbe-908e-5601948312d2-kube-api-access-jzckm" (OuterVolumeSpecName: "kube-api-access-jzckm") pod "d4441b18-67b8-4fbe-908e-5601948312d2" (UID: "d4441b18-67b8-4fbe-908e-5601948312d2"). InnerVolumeSpecName "kube-api-access-jzckm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:28:56.057841 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.057780    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6bd9043c-0b68-428d-b2a6-f5bed707a7a0" (UID: "6bd9043c-0b68-428d-b2a6-f5bed707a7a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:28:56.156676 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.156655    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6wkv\" (UniqueName: \"kubernetes.io/projected/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-kube-api-access-v6wkv\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:28:56.156676 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.156675    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzckm\" (UniqueName: \"kubernetes.io/projected/d4441b18-67b8-4fbe-908e-5601948312d2-kube-api-access-jzckm\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:28:56.156803 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.156684    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:28:56.156803 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.156694    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bd9043c-0b68-428d-b2a6-f5bed707a7a0-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:28:56.156803 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.156702    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4441b18-67b8-4fbe-908e-5601948312d2-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:28:56.501336 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.501254    2560 generic.go:356] "Generic (PLEG): container finished" podID="d4441b18-67b8-4fbe-908e-5601948312d2" containerID="88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d" exitCode=0
Jul 26 16:28:56.501336 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.501307    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"
Jul 26 16:28:56.501556 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.501298    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp" event={"ID":"d4441b18-67b8-4fbe-908e-5601948312d2","Type":"ContainerDied","Data":"88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d"}
Jul 26 16:28:56.501556 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.501456    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp" event={"ID":"d4441b18-67b8-4fbe-908e-5601948312d2","Type":"ContainerDied","Data":"591d2702be7c144616c85f9c98032f2c5980041e1626406789cb33e3a6b0b6db"}
Jul 26 16:28:56.501556 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.501495    2560 scope.go:117] "RemoveContainer" containerID="88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d"
Jul 26 16:28:56.502823 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.502802    2560 generic.go:356] "Generic (PLEG): container finished" podID="6bd9043c-0b68-428d-b2a6-f5bed707a7a0" containerID="a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d" exitCode=0
Jul 26 16:28:56.502960 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.502847    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"
Jul 26 16:28:56.502960 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.502909    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9" event={"ID":"6bd9043c-0b68-428d-b2a6-f5bed707a7a0","Type":"ContainerDied","Data":"a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d"}
Jul 26 16:28:56.502960 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.502957    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9" event={"ID":"6bd9043c-0b68-428d-b2a6-f5bed707a7a0","Type":"ContainerDied","Data":"fa7857d65dc7948c82240f51ce248280e8eabcee86c25c533a8305d59cca3e3e"}
Jul 26 16:28:56.511354 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.511333    2560 scope.go:117] "RemoveContainer" containerID="88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d"
Jul 26 16:28:56.512068 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:28:56.512044    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d\": container with ID starting with 88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d not found: ID does not exist" containerID="88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d"
Jul 26 16:28:56.512153 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.512080    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d"} err="failed to get container status \"88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d\": rpc error: code = NotFound desc = could not find container \"88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d\": container with ID starting with 88201d04fb4bc61dbd56a14dcd1cb0a7d4b4d4df972e09e8c4f7396b4e1a123d not found: ID does not exist"
Jul 26 16:28:56.512153 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.512097    2560 scope.go:117] "RemoveContainer" containerID="a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d"
Jul 26 16:28:56.519836 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.519820    2560 scope.go:117] "RemoveContainer" containerID="a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d"
Jul 26 16:28:56.520085 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:28:56.520067    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d\": container with ID starting with a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d not found: ID does not exist" containerID="a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d"
Jul 26 16:28:56.520129 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.520093    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d"} err="failed to get container status \"a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d\": rpc error: code = NotFound desc = could not find container \"a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d\": container with ID starting with a8979c62217de2a23ae9995bfb1068ff5cfe6f2e684b6b78b5c38654d73c7d2d not found: ID does not exist"
Jul 26 16:28:56.534758 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.534737    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"]
Jul 26 16:28:56.545403 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.545385    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8c3dba-68b5dbdc9b-h74m9"]
Jul 26 16:28:56.562451 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.562429    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"]
Jul 26 16:28:56.577328 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:56.577307    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c3dba-79c444c74-6lzkp"]
Jul 26 16:28:57.139098 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:57.139069    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd9043c-0b68-428d-b2a6-f5bed707a7a0" path="/var/lib/kubelet/pods/6bd9043c-0b68-428d-b2a6-f5bed707a7a0/volumes"
Jul 26 16:28:57.139458 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:28:57.139424    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4441b18-67b8-4fbe-908e-5601948312d2" path="/var/lib/kubelet/pods/d4441b18-67b8-4fbe-908e-5601948312d2/volumes"
Jul 26 16:29:04.613689 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.613656    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wpgdj"]
Jul 26 16:29:04.614099 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614056    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4d9ecf0-c007-43a1-9a89-252159f035a9" containerName="payload-processing"
Jul 26 16:29:04.614099 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614068    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d9ecf0-c007-43a1-9a89-252159f035a9" containerName="payload-processing"
Jul 26 16:29:04.614099 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614080    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="170549ed-2881-4740-bb8c-4b6ec7b0373f" containerName="maas-api"
Jul 26 16:29:04.614099 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614086    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="170549ed-2881-4740-bb8c-4b6ec7b0373f" containerName="maas-api"
Jul 26 16:29:04.614099 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614094    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4441b18-67b8-4fbe-908e-5601948312d2" containerName="payload-processing"
Jul 26 16:29:04.614099 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614100    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4441b18-67b8-4fbe-908e-5601948312d2" containerName="payload-processing"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614108    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38403683-79f4-424b-9503-fdeca74593b4" containerName="revoke-keys"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614113    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38403683-79f4-424b-9503-fdeca74593b4" containerName="revoke-keys"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614131    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd9043c-0b68-428d-b2a6-f5bed707a7a0" containerName="maas-api"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614137    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd9043c-0b68-428d-b2a6-f5bed707a7a0" containerName="maas-api"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614145    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cc2790d-a727-4593-83e1-f94511c4eeb9" containerName="payload-pre-processing"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614150    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cc2790d-a727-4593-83e1-f94511c4eeb9" containerName="payload-pre-processing"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614156    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b55f70f2-a512-4639-922c-4ed9c2cd8ff6" containerName="maas-api"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614161    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b55f70f2-a512-4639-922c-4ed9c2cd8ff6" containerName="maas-api"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614216    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cc2790d-a727-4593-83e1-f94511c4eeb9" containerName="payload-pre-processing"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614224    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b55f70f2-a512-4639-922c-4ed9c2cd8ff6" containerName="maas-api"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614230    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4d9ecf0-c007-43a1-9a89-252159f035a9" containerName="payload-processing"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614236    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="170549ed-2881-4740-bb8c-4b6ec7b0373f" containerName="maas-api"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614243    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4441b18-67b8-4fbe-908e-5601948312d2" containerName="payload-processing"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614249    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd9043c-0b68-428d-b2a6-f5bed707a7a0" containerName="maas-api"
Jul 26 16:29:04.614324 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.614255    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38403683-79f4-424b-9503-fdeca74593b4" containerName="revoke-keys"
Jul 26 16:29:04.618765 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.618745    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:04.620744 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.620724    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hcf6p\""
Jul 26 16:29:04.626193 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.626162    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wpgdj"]
Jul 26 16:29:04.719764 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.719736    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xg8k\" (UniqueName: \"kubernetes.io/projected/39533beb-c710-4f71-b669-790299cfb866-kube-api-access-4xg8k\") pod \"kuadrant-operator-catalog-wpgdj\" (UID: \"39533beb-c710-4f71-b669-790299cfb866\") " pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:04.820498 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.820471    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xg8k\" (UniqueName: \"kubernetes.io/projected/39533beb-c710-4f71-b669-790299cfb866-kube-api-access-4xg8k\") pod \"kuadrant-operator-catalog-wpgdj\" (UID: \"39533beb-c710-4f71-b669-790299cfb866\") " pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:04.828585 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.828551    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xg8k\" (UniqueName: \"kubernetes.io/projected/39533beb-c710-4f71-b669-790299cfb866-kube-api-access-4xg8k\") pod \"kuadrant-operator-catalog-wpgdj\" (UID: \"39533beb-c710-4f71-b669-790299cfb866\") " pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:04.932291 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.932269    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hcf6p\""
Jul 26 16:29:04.941147 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:04.941125    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:05.066554 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:05.066531    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wpgdj"]
Jul 26 16:29:05.068527 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:29:05.068495    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39533beb_c710_4f71_b669_790299cfb866.slice/crio-6d0b9bf9e5446b4a76e5527384c138067c3136254d4a0ba834f0ee87dab894f8 WatchSource:0}: Error finding container 6d0b9bf9e5446b4a76e5527384c138067c3136254d4a0ba834f0ee87dab894f8: Status 404 returned error can't find the container with id 6d0b9bf9e5446b4a76e5527384c138067c3136254d4a0ba834f0ee87dab894f8
Jul 26 16:29:05.549507 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:05.549478    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj" event={"ID":"39533beb-c710-4f71-b669-790299cfb866","Type":"ContainerStarted","Data":"6d0b9bf9e5446b4a76e5527384c138067c3136254d4a0ba834f0ee87dab894f8"}
Jul 26 16:29:06.288679 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:06.288648    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"]
Jul 26 16:29:06.292437 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:06.292418    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"
Jul 26 16:29:06.294560 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:06.294535    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:29:06.300159 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:06.300105    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"]
Jul 26 16:29:06.437379 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:06.437348    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a752446f-32cc-47b5-a425-5d4b5d662462-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74\" (UID: \"a752446f-32cc-47b5-a425-5d4b5d662462\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"
Jul 26 16:29:06.538347 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:06.538310    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a752446f-32cc-47b5-a425-5d4b5d662462-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74\" (UID: \"a752446f-32cc-47b5-a425-5d4b5d662462\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"
Jul 26 16:29:06.539051 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:06.538986    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a752446f-32cc-47b5-a425-5d4b5d662462-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74\" (UID: \"a752446f-32cc-47b5-a425-5d4b5d662462\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"
Jul 26 16:29:06.605398 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:06.605368    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"
Jul 26 16:29:06.955647 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:06.955625    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"]
Jul 26 16:29:06.962250 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:29:06.962207    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda752446f_32cc_47b5_a425_5d4b5d662462.slice/crio-8335870f1aea1fd5a2e2d3dc88b5466428b262952c708f5b63dd88594a9b5ee3 WatchSource:0}: Error finding container 8335870f1aea1fd5a2e2d3dc88b5466428b262952c708f5b63dd88594a9b5ee3: Status 404 returned error can't find the container with id 8335870f1aea1fd5a2e2d3dc88b5466428b262952c708f5b63dd88594a9b5ee3
Jul 26 16:29:07.560964 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:07.560929    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj" event={"ID":"39533beb-c710-4f71-b669-790299cfb866","Type":"ContainerStarted","Data":"7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808"}
Jul 26 16:29:07.562490 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:07.562467    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74" event={"ID":"a752446f-32cc-47b5-a425-5d4b5d662462","Type":"ContainerStarted","Data":"5557019fcba7a28c95ae63aaaf718406e33562f7244217568d20093f409b4b56"}
Jul 26 16:29:07.562599 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:07.562508    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74" event={"ID":"a752446f-32cc-47b5-a425-5d4b5d662462","Type":"ContainerStarted","Data":"8335870f1aea1fd5a2e2d3dc88b5466428b262952c708f5b63dd88594a9b5ee3"}
Jul 26 16:29:07.577947 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:07.577878    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj" podStartSLOduration=1.767259688 podStartE2EDuration="3.57786466s" totalImagesPullingTime="1.810604972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:29:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:29:07.57605616 +0000 UTC m=+3063.075445999" watchObservedRunningTime="2026-07-26 16:29:07.57786466 +0000 UTC m=+3063.077254488"
Jul 26 16:29:07.591429 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:07.591385    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74" podStartSLOduration=1.5913687300000001 podStartE2EDuration="1.59136873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:29:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:29:07.590655405 +0000 UTC m=+3063.090045242" watchObservedRunningTime="2026-07-26 16:29:07.59136873 +0000 UTC m=+3063.090758572"
Jul 26 16:29:08.568129 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:08.568090    2560 generic.go:356] "Generic (PLEG): container finished" podID="a752446f-32cc-47b5-a425-5d4b5d662462" containerID="5557019fcba7a28c95ae63aaaf718406e33562f7244217568d20093f409b4b56" exitCode=0
Jul 26 16:29:08.568548 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:08.568184    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74" event={"ID":"a752446f-32cc-47b5-a425-5d4b5d662462","Type":"ContainerDied","Data":"5557019fcba7a28c95ae63aaaf718406e33562f7244217568d20093f409b4b56"}
Jul 26 16:29:09.705338 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:09.705314    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"
Jul 26 16:29:09.864986 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:09.864912    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a752446f-32cc-47b5-a425-5d4b5d662462-maas-api-service-ca\") pod \"a752446f-32cc-47b5-a425-5d4b5d662462\" (UID: \"a752446f-32cc-47b5-a425-5d4b5d662462\") "
Jul 26 16:29:09.865315 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:09.865283    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a752446f-32cc-47b5-a425-5d4b5d662462-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a752446f-32cc-47b5-a425-5d4b5d662462" (UID: "a752446f-32cc-47b5-a425-5d4b5d662462"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:29:09.965541 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:09.965507    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a752446f-32cc-47b5-a425-5d4b5d662462-maas-api-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:10.578279 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:10.578241    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"
Jul 26 16:29:10.578443 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:10.578280    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74" event={"ID":"a752446f-32cc-47b5-a425-5d4b5d662462","Type":"ContainerDied","Data":"8335870f1aea1fd5a2e2d3dc88b5466428b262952c708f5b63dd88594a9b5ee3"}
Jul 26 16:29:10.578443 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:10.578328    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8335870f1aea1fd5a2e2d3dc88b5466428b262952c708f5b63dd88594a9b5ee3"
Jul 26 16:29:14.941454 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:14.941421    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:14.941867 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:14.941640    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:14.962775 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:14.962749    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:15.623355 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:15.623330    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:15.659208 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:15.659172    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wpgdj"]
Jul 26 16:29:16.365384 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.365353    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"]
Jul 26 16:29:16.365838 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.365640    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x" podUID="e13d5469-5380-4b13-97dc-9a5ff72adcd6" containerName="maas-api" containerID="cri-o://2afeb590f7ca54fe2b0e9c8cbe508547bda62e103c97ce0d52f972ace44da61b" gracePeriod=30
Jul 26 16:29:16.416118 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.416087    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"]
Jul 26 16:29:16.416341 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.416300    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj" podUID="cbfd5177-b27f-4771-8c24-80e86645aa39" containerName="payload-processing" containerID="cri-o://06d4c988045456793cde791c2fae111c7cc58e8d704eb10394c3b75eae3d5828" gracePeriod=30
Jul 26 16:29:16.422686 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.422135    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"]
Jul 26 16:29:16.422686 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.422354    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266" podUID="66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8" containerName="payload-pre-processing" containerID="cri-o://6b00e710129e67858346f2c61fde785cec39ea5ad2057583d9d7c978b225559d" gracePeriod=30
Jul 26 16:29:16.610915 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.610850    2560 generic.go:356] "Generic (PLEG): container finished" podID="cbfd5177-b27f-4771-8c24-80e86645aa39" containerID="06d4c988045456793cde791c2fae111c7cc58e8d704eb10394c3b75eae3d5828" exitCode=0
Jul 26 16:29:16.611133 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.610983    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj" event={"ID":"cbfd5177-b27f-4771-8c24-80e86645aa39","Type":"ContainerDied","Data":"06d4c988045456793cde791c2fae111c7cc58e8d704eb10394c3b75eae3d5828"}
Jul 26 16:29:16.615947 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.615392    2560 generic.go:356] "Generic (PLEG): container finished" podID="66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8" containerID="6b00e710129e67858346f2c61fde785cec39ea5ad2057583d9d7c978b225559d" exitCode=0
Jul 26 16:29:16.615947 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.615472    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266" event={"ID":"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8","Type":"ContainerDied","Data":"6b00e710129e67858346f2c61fde785cec39ea5ad2057583d9d7c978b225559d"}
Jul 26 16:29:16.617181 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.617156    2560 generic.go:356] "Generic (PLEG): container finished" podID="e13d5469-5380-4b13-97dc-9a5ff72adcd6" containerID="2afeb590f7ca54fe2b0e9c8cbe508547bda62e103c97ce0d52f972ace44da61b" exitCode=0
Jul 26 16:29:16.617422 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.617394    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x" event={"ID":"e13d5469-5380-4b13-97dc-9a5ff72adcd6","Type":"ContainerDied","Data":"2afeb590f7ca54fe2b0e9c8cbe508547bda62e103c97ce0d52f972ace44da61b"}
Jul 26 16:29:16.717028 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.717004    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:29:16.722484 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.722464    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:29:16.738857 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.738835    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:29:16.818633 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.818602    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbfd5177-b27f-4771-8c24-80e86645aa39-plugins-config-volume\") pod \"cbfd5177-b27f-4771-8c24-80e86645aa39\" (UID: \"cbfd5177-b27f-4771-8c24-80e86645aa39\") "
Jul 26 16:29:16.818774 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.818650    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdxkg\" (UniqueName: \"kubernetes.io/projected/cbfd5177-b27f-4771-8c24-80e86645aa39-kube-api-access-hdxkg\") pod \"cbfd5177-b27f-4771-8c24-80e86645aa39\" (UID: \"cbfd5177-b27f-4771-8c24-80e86645aa39\") "
Jul 26 16:29:16.818774 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.818678    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13d5469-5380-4b13-97dc-9a5ff72adcd6-maas-api-tls\") pod \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") "
Jul 26 16:29:16.818774 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.818764    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s67vq\" (UniqueName: \"kubernetes.io/projected/e13d5469-5380-4b13-97dc-9a5ff72adcd6-kube-api-access-s67vq\") pod \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") "
Jul 26 16:29:16.818923 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.818810    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e13d5469-5380-4b13-97dc-9a5ff72adcd6-openshift-service-ca\") pod \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\" (UID: \"e13d5469-5380-4b13-97dc-9a5ff72adcd6\") "
Jul 26 16:29:16.819188 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.819156    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbfd5177-b27f-4771-8c24-80e86645aa39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cbfd5177-b27f-4771-8c24-80e86645aa39" (UID: "cbfd5177-b27f-4771-8c24-80e86645aa39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:29:16.819188 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.819178    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e13d5469-5380-4b13-97dc-9a5ff72adcd6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e13d5469-5380-4b13-97dc-9a5ff72adcd6" (UID: "e13d5469-5380-4b13-97dc-9a5ff72adcd6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:29:16.820738 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.820718    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e13d5469-5380-4b13-97dc-9a5ff72adcd6-kube-api-access-s67vq" (OuterVolumeSpecName: "kube-api-access-s67vq") pod "e13d5469-5380-4b13-97dc-9a5ff72adcd6" (UID: "e13d5469-5380-4b13-97dc-9a5ff72adcd6"). InnerVolumeSpecName "kube-api-access-s67vq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:29:16.821169 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.821153    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbfd5177-b27f-4771-8c24-80e86645aa39-kube-api-access-hdxkg" (OuterVolumeSpecName: "kube-api-access-hdxkg") pod "cbfd5177-b27f-4771-8c24-80e86645aa39" (UID: "cbfd5177-b27f-4771-8c24-80e86645aa39"). InnerVolumeSpecName "kube-api-access-hdxkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:29:16.821218 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.821164    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e13d5469-5380-4b13-97dc-9a5ff72adcd6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e13d5469-5380-4b13-97dc-9a5ff72adcd6" (UID: "e13d5469-5380-4b13-97dc-9a5ff72adcd6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:29:16.920147 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.920125    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kndz\" (UniqueName: \"kubernetes.io/projected/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-kube-api-access-6kndz\") pod \"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8\" (UID: \"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8\") "
Jul 26 16:29:16.920268 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.920167    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-plugins-config-volume\") pod \"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8\" (UID: \"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8\") "
Jul 26 16:29:16.920343 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.920329    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbfd5177-b27f-4771-8c24-80e86645aa39-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:16.920405 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.920344    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdxkg\" (UniqueName: \"kubernetes.io/projected/cbfd5177-b27f-4771-8c24-80e86645aa39-kube-api-access-hdxkg\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:16.920405 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.920353    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13d5469-5380-4b13-97dc-9a5ff72adcd6-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:16.920405 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.920361    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s67vq\" (UniqueName: \"kubernetes.io/projected/e13d5469-5380-4b13-97dc-9a5ff72adcd6-kube-api-access-s67vq\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:16.920405 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.920370    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e13d5469-5380-4b13-97dc-9a5ff72adcd6-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:16.920612 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.920562    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8" (UID: "66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:29:16.921769 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:16.921750    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-kube-api-access-6kndz" (OuterVolumeSpecName: "kube-api-access-6kndz") pod "66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8" (UID: "66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8"). InnerVolumeSpecName "kube-api-access-6kndz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:29:17.020948 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.020923    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:17.020948 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.020947    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kndz\" (UniqueName: \"kubernetes.io/projected/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8-kube-api-access-6kndz\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:17.622872 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.622837    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj" event={"ID":"cbfd5177-b27f-4771-8c24-80e86645aa39","Type":"ContainerDied","Data":"b5d2b6d9764b445d3ab85e48e205e307030ac17fda16af69e0894bf4c654ba9d"}
Jul 26 16:29:17.623306 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.622853    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"
Jul 26 16:29:17.623306 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.622918    2560 scope.go:117] "RemoveContainer" containerID="06d4c988045456793cde791c2fae111c7cc58e8d704eb10394c3b75eae3d5828"
Jul 26 16:29:17.624396 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.624379    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"
Jul 26 16:29:17.624494 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.624400    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266" event={"ID":"66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8","Type":"ContainerDied","Data":"9df3e0a7dfda9f67684fdf301178bccfe866cca6c500792c9f561105e44df0fb"}
Jul 26 16:29:17.626237 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.626199    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"
Jul 26 16:29:17.626237 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.626209    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj" podUID="39533beb-c710-4f71-b669-790299cfb866" containerName="registry-server" containerID="cri-o://7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808" gracePeriod=2
Jul 26 16:29:17.626438 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.626322    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x" event={"ID":"e13d5469-5380-4b13-97dc-9a5ff72adcd6","Type":"ContainerDied","Data":"107705b82757d813dd76562847331d3f11f29105085f7f1afbb09f35fc40819b"}
Jul 26 16:29:17.633807 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.633788    2560 scope.go:117] "RemoveContainer" containerID="6b00e710129e67858346f2c61fde785cec39ea5ad2057583d9d7c978b225559d"
Jul 26 16:29:17.658762 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.658738    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"]
Jul 26 16:29:17.660501 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.660480    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aadfb2-595d6dcb6b-4r62x"]
Jul 26 16:29:17.673363 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.673341    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"]
Jul 26 16:29:17.674557 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.674527    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-aadfb2-c88cd7b78-lqkkj"]
Jul 26 16:29:17.683091 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.683069    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"]
Jul 26 16:29:17.687055 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.687036    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-aadfb2-5cb4dcdcf6-zc266"]
Jul 26 16:29:17.690178 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.690158    2560 scope.go:117] "RemoveContainer" containerID="2afeb590f7ca54fe2b0e9c8cbe508547bda62e103c97ce0d52f972ace44da61b"
Jul 26 16:29:17.874810 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:17.874783    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:18.028178 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.028143    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xg8k\" (UniqueName: \"kubernetes.io/projected/39533beb-c710-4f71-b669-790299cfb866-kube-api-access-4xg8k\") pod \"39533beb-c710-4f71-b669-790299cfb866\" (UID: \"39533beb-c710-4f71-b669-790299cfb866\") "
Jul 26 16:29:18.030280 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.030246    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39533beb-c710-4f71-b669-790299cfb866-kube-api-access-4xg8k" (OuterVolumeSpecName: "kube-api-access-4xg8k") pod "39533beb-c710-4f71-b669-790299cfb866" (UID: "39533beb-c710-4f71-b669-790299cfb866"). InnerVolumeSpecName "kube-api-access-4xg8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:29:18.128738 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.128712    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xg8k\" (UniqueName: \"kubernetes.io/projected/39533beb-c710-4f71-b669-790299cfb866-kube-api-access-4xg8k\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:18.635045 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.635013    2560 generic.go:356] "Generic (PLEG): container finished" podID="39533beb-c710-4f71-b669-790299cfb866" containerID="7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808" exitCode=0
Jul 26 16:29:18.635471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.635055    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj"
Jul 26 16:29:18.635471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.635057    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj" event={"ID":"39533beb-c710-4f71-b669-790299cfb866","Type":"ContainerDied","Data":"7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808"}
Jul 26 16:29:18.635471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.635118    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wpgdj" event={"ID":"39533beb-c710-4f71-b669-790299cfb866","Type":"ContainerDied","Data":"6d0b9bf9e5446b4a76e5527384c138067c3136254d4a0ba834f0ee87dab894f8"}
Jul 26 16:29:18.635471 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.635149    2560 scope.go:117] "RemoveContainer" containerID="7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808"
Jul 26 16:29:18.643658 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.643641    2560 scope.go:117] "RemoveContainer" containerID="7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808"
Jul 26 16:29:18.643948 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:29:18.643922    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808\": container with ID starting with 7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808 not found: ID does not exist" containerID="7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808"
Jul 26 16:29:18.644042 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.643960    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808"} err="failed to get container status \"7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808\": rpc error: code = NotFound desc = could not find container \"7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808\": container with ID starting with 7abf74fbb9cd21d8ef36b670eae38024adabc9727c5430a706dbc7e8e68df808 not found: ID does not exist"
Jul 26 16:29:18.654989 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.654965    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wpgdj"]
Jul 26 16:29:18.663370 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:18.663351    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wpgdj"]
Jul 26 16:29:19.138817 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:19.138779    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39533beb-c710-4f71-b669-790299cfb866" path="/var/lib/kubelet/pods/39533beb-c710-4f71-b669-790299cfb866/volumes"
Jul 26 16:29:19.139237 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:19.139216    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8" path="/var/lib/kubelet/pods/66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8/volumes"
Jul 26 16:29:19.139602 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:19.139585    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbfd5177-b27f-4771-8c24-80e86645aa39" path="/var/lib/kubelet/pods/cbfd5177-b27f-4771-8c24-80e86645aa39/volumes"
Jul 26 16:29:19.139997 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:19.139981    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e13d5469-5380-4b13-97dc-9a5ff72adcd6" path="/var/lib/kubelet/pods/e13d5469-5380-4b13-97dc-9a5ff72adcd6/volumes"
Jul 26 16:29:21.382984 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:21.382950    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"]
Jul 26 16:29:21.385576 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:21.385554    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aadfb2-bwz74"]
Jul 26 16:29:23.143074 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:23.139776    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a752446f-32cc-47b5-a425-5d4b5d662462" path="/var/lib/kubelet/pods/a752446f-32cc-47b5-a425-5d4b5d662462/volumes"
Jul 26 16:29:39.534083 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.534043    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"]
Jul 26 16:29:39.534625 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.534286    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs" podUID="05369022-95a6-403e-a1e2-0811151da7ba" containerName="maas-api" containerID="cri-o://6266c434c20be5ad8d6300525fc499f92bc87192e7b28b8303250bb39ef028a0" gracePeriod=30
Jul 26 16:29:39.621106 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.621075    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"]
Jul 26 16:29:39.621338 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.621308    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n" podUID="a4f039fb-3af1-4197-8f87-f564f47f771a" containerName="payload-processing" containerID="cri-o://b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2" gracePeriod=30
Jul 26 16:29:39.643440 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.643416    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"]
Jul 26 16:29:39.643643 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.643606    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9" podUID="4949ccdd-82f7-4a9f-88a7-fb9d96ec5382" containerName="payload-pre-processing" containerID="cri-o://9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e" gracePeriod=30
Jul 26 16:29:39.733085 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.733060    2560 generic.go:356] "Generic (PLEG): container finished" podID="05369022-95a6-403e-a1e2-0811151da7ba" containerID="6266c434c20be5ad8d6300525fc499f92bc87192e7b28b8303250bb39ef028a0" exitCode=0
Jul 26 16:29:39.733190 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.733138    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs" event={"ID":"05369022-95a6-403e-a1e2-0811151da7ba","Type":"ContainerDied","Data":"6266c434c20be5ad8d6300525fc499f92bc87192e7b28b8303250bb39ef028a0"}
Jul 26 16:29:39.885569 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.885539    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:29:39.929090 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.929064    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:29:39.939802 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.939783    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:29:39.996969 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.996939    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkb6k\" (UniqueName: \"kubernetes.io/projected/a4f039fb-3af1-4197-8f87-f564f47f771a-kube-api-access-rkb6k\") pod \"a4f039fb-3af1-4197-8f87-f564f47f771a\" (UID: \"a4f039fb-3af1-4197-8f87-f564f47f771a\") "
Jul 26 16:29:39.997127 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.996979    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-plugins-config-volume\") pod \"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382\" (UID: \"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382\") "
Jul 26 16:29:39.997127 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.997016    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05369022-95a6-403e-a1e2-0811151da7ba-openshift-service-ca\") pod \"05369022-95a6-403e-a1e2-0811151da7ba\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") "
Jul 26 16:29:39.997127 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.997113    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6262w\" (UniqueName: \"kubernetes.io/projected/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-kube-api-access-6262w\") pod \"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382\" (UID: \"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382\") "
Jul 26 16:29:39.997289 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.997146    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4f039fb-3af1-4197-8f87-f564f47f771a-plugins-config-volume\") pod \"a4f039fb-3af1-4197-8f87-f564f47f771a\" (UID: \"a4f039fb-3af1-4197-8f87-f564f47f771a\") "
Jul 26 16:29:39.997289 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.997175    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05369022-95a6-403e-a1e2-0811151da7ba-maas-api-tls\") pod \"05369022-95a6-403e-a1e2-0811151da7ba\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") "
Jul 26 16:29:39.997289 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.997217    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzfxm\" (UniqueName: \"kubernetes.io/projected/05369022-95a6-403e-a1e2-0811151da7ba-kube-api-access-qzfxm\") pod \"05369022-95a6-403e-a1e2-0811151da7ba\" (UID: \"05369022-95a6-403e-a1e2-0811151da7ba\") "
Jul 26 16:29:39.997498 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.997346    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4949ccdd-82f7-4a9f-88a7-fb9d96ec5382" (UID: "4949ccdd-82f7-4a9f-88a7-fb9d96ec5382"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:29:39.997629 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.997456    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05369022-95a6-403e-a1e2-0811151da7ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "05369022-95a6-403e-a1e2-0811151da7ba" (UID: "05369022-95a6-403e-a1e2-0811151da7ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:29:39.997629 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.997542    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4f039fb-3af1-4197-8f87-f564f47f771a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a4f039fb-3af1-4197-8f87-f564f47f771a" (UID: "a4f039fb-3af1-4197-8f87-f564f47f771a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:29:39.999301 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.999275    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-kube-api-access-6262w" (OuterVolumeSpecName: "kube-api-access-6262w") pod "4949ccdd-82f7-4a9f-88a7-fb9d96ec5382" (UID: "4949ccdd-82f7-4a9f-88a7-fb9d96ec5382"). InnerVolumeSpecName "kube-api-access-6262w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:29:39.999301 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.999282    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05369022-95a6-403e-a1e2-0811151da7ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "05369022-95a6-403e-a1e2-0811151da7ba" (UID: "05369022-95a6-403e-a1e2-0811151da7ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:29:39.999429 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.999306    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4f039fb-3af1-4197-8f87-f564f47f771a-kube-api-access-rkb6k" (OuterVolumeSpecName: "kube-api-access-rkb6k") pod "a4f039fb-3af1-4197-8f87-f564f47f771a" (UID: "a4f039fb-3af1-4197-8f87-f564f47f771a"). InnerVolumeSpecName "kube-api-access-rkb6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:29:39.999429 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:39.999388    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05369022-95a6-403e-a1e2-0811151da7ba-kube-api-access-qzfxm" (OuterVolumeSpecName: "kube-api-access-qzfxm") pod "05369022-95a6-403e-a1e2-0811151da7ba" (UID: "05369022-95a6-403e-a1e2-0811151da7ba"). InnerVolumeSpecName "kube-api-access-qzfxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:29:40.098556 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.098502    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4f039fb-3af1-4197-8f87-f564f47f771a-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:40.098556 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.098522    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05369022-95a6-403e-a1e2-0811151da7ba-maas-api-tls\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:40.098556 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.098532    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzfxm\" (UniqueName: \"kubernetes.io/projected/05369022-95a6-403e-a1e2-0811151da7ba-kube-api-access-qzfxm\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:40.098556 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.098541    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkb6k\" (UniqueName: \"kubernetes.io/projected/a4f039fb-3af1-4197-8f87-f564f47f771a-kube-api-access-rkb6k\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:40.098556 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.098550    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-plugins-config-volume\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:40.098556 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.098558    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05369022-95a6-403e-a1e2-0811151da7ba-openshift-service-ca\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:40.098805 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.098568    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6262w\" (UniqueName: \"kubernetes.io/projected/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382-kube-api-access-6262w\") on node \"ip-10-0-132-10.ec2.internal\" DevicePath \"\""
Jul 26 16:29:40.739432 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.739395    2560 generic.go:356] "Generic (PLEG): container finished" podID="a4f039fb-3af1-4197-8f87-f564f47f771a" containerID="b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2" exitCode=0
Jul 26 16:29:40.739432 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.739440    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"
Jul 26 16:29:40.739956 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.739486    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n" event={"ID":"a4f039fb-3af1-4197-8f87-f564f47f771a","Type":"ContainerDied","Data":"b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2"}
Jul 26 16:29:40.739956 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.739542    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n" event={"ID":"a4f039fb-3af1-4197-8f87-f564f47f771a","Type":"ContainerDied","Data":"acd368987459ea4ccf6c8c870873fe361950273c00af6cd9fc9ad6470bd09211"}
Jul 26 16:29:40.739956 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.739568    2560 scope.go:117] "RemoveContainer" containerID="b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2"
Jul 26 16:29:40.740955 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.740918    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs" event={"ID":"05369022-95a6-403e-a1e2-0811151da7ba","Type":"ContainerDied","Data":"9e674e887db066f80e313cb3a490b402af582dacb7955ed8ea8f3d8315c12a64"}
Jul 26 16:29:40.740955 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.740936    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"
Jul 26 16:29:40.742481 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.742448    2560 generic.go:356] "Generic (PLEG): container finished" podID="4949ccdd-82f7-4a9f-88a7-fb9d96ec5382" containerID="9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e" exitCode=0
Jul 26 16:29:40.742587 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.742500    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"
Jul 26 16:29:40.742587 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.742533    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9" event={"ID":"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382","Type":"ContainerDied","Data":"9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e"}
Jul 26 16:29:40.742707 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.742594    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9" event={"ID":"4949ccdd-82f7-4a9f-88a7-fb9d96ec5382","Type":"ContainerDied","Data":"fe51f1efaf20b9a789b301b39bbb03d8c8e9c5e6b59349d9ce333a747f7fb510"}
Jul 26 16:29:40.750200 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.750180    2560 scope.go:117] "RemoveContainer" containerID="b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2"
Jul 26 16:29:40.750501 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:29:40.750473    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2\": container with ID starting with b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2 not found: ID does not exist" containerID="b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2"
Jul 26 16:29:40.750581 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.750511    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2"} err="failed to get container status \"b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2\": rpc error: code = NotFound desc = could not find container \"b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2\": container with ID starting with b372fa724b804ce3bb60ebc6156d1cea96d4fc20e7855dd02e19018eb1214eb2 not found: ID does not exist"
Jul 26 16:29:40.750581 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.750535    2560 scope.go:117] "RemoveContainer" containerID="6266c434c20be5ad8d6300525fc499f92bc87192e7b28b8303250bb39ef028a0"
Jul 26 16:29:40.758686 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.758669    2560 scope.go:117] "RemoveContainer" containerID="9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e"
Jul 26 16:29:40.760487 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.760468    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"]
Jul 26 16:29:40.764030 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.764011    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e0208c-5476cd5f94-w678n"]
Jul 26 16:29:40.766432 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.766415    2560 scope.go:117] "RemoveContainer" containerID="9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e"
Jul 26 16:29:40.766700 ip-10-0-132-10 kubenswrapper[2560]: E0726 16:29:40.766683    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e\": container with ID starting with 9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e not found: ID does not exist" containerID="9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e"
Jul 26 16:29:40.766743 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.766707    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e"} err="failed to get container status \"9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e\": rpc error: code = NotFound desc = could not find container \"9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e\": container with ID starting with 9ea0308e5a59863f534e251643b6e0fbc248e51e54dea92dc40b798b8318c14e not found: ID does not exist"
Jul 26 16:29:40.771643 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.771624    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"]
Jul 26 16:29:40.774224 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.774204    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e0208c-6f687c646d-f74k9"]
Jul 26 16:29:40.783117 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.783100    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"]
Jul 26 16:29:40.786647 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:40.786628    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e0208c-7bb67786d7-g48rs"]
Jul 26 16:29:41.138022 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:41.137996    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05369022-95a6-403e-a1e2-0811151da7ba" path="/var/lib/kubelet/pods/05369022-95a6-403e-a1e2-0811151da7ba/volumes"
Jul 26 16:29:41.138339 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:41.138327    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4949ccdd-82f7-4a9f-88a7-fb9d96ec5382" path="/var/lib/kubelet/pods/4949ccdd-82f7-4a9f-88a7-fb9d96ec5382/volumes"
Jul 26 16:29:41.138625 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:41.138614    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4f039fb-3af1-4197-8f87-f564f47f771a" path="/var/lib/kubelet/pods/a4f039fb-3af1-4197-8f87-f564f47f771a/volumes"
Jul 26 16:29:56.215779 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:56.215748    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qmwhg_49024e42-a310-4da5-b905-cb44535548a0/manager/0.log"
Jul 26 16:29:56.337846 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:56.337820    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sgtxq_a1ef07ad-8d81-4dc9-9f94-160eef929d4f/manager/0.log"
Jul 26 16:29:56.466453 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:56.466387    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86d86cd99-sm5c5_9e01530d-9d12-4e71-b9d1-a311b268f572/manager/0.log"
Jul 26 16:29:56.907644 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:56.907620    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d4685587-6w2v4_a3d0b25c-4d79-4def-99fc-8530a60fc98a/copy-manifests/0.log"
Jul 26 16:29:56.925958 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:56.925768    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d4685587-6w2v4_a3d0b25c-4d79-4def-99fc-8530a60fc98a/manager/0.log"
Jul 26 16:29:57.404561 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:57.404534    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c56dc8ff8-pz8zn_9c444d25-e535-4332-9d97-14733da7bdcc/maas-api/0.log"
Jul 26 16:29:57.730945 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:57.730861    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-pd89t_759acf50-d889-44a2-94c4-f20d52ec3b1f/postgres/0.log"
Jul 26 16:29:59.816349 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:29:59.816321    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cgq5h_dfe51eeb-0d36-455a-a8e7-63e1d4ec1f99/manager/0.log"
Jul 26 16:30:00.204378 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204350    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk"]
Jul 26 16:30:00.204661 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204648    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8" containerName="payload-pre-processing"
Jul 26 16:30:00.204710 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204662    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8" containerName="payload-pre-processing"
Jul 26 16:30:00.204710 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204672    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4949ccdd-82f7-4a9f-88a7-fb9d96ec5382" containerName="payload-pre-processing"
Jul 26 16:30:00.204710 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204678    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4949ccdd-82f7-4a9f-88a7-fb9d96ec5382" containerName="payload-pre-processing"
Jul 26 16:30:00.204710 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204688    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e13d5469-5380-4b13-97dc-9a5ff72adcd6" containerName="maas-api"
Jul 26 16:30:00.204710 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204694    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e13d5469-5380-4b13-97dc-9a5ff72adcd6" containerName="maas-api"
Jul 26 16:30:00.204710 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204702    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a752446f-32cc-47b5-a425-5d4b5d662462" containerName="revoke-keys"
Jul 26 16:30:00.204710 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204708    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a752446f-32cc-47b5-a425-5d4b5d662462" containerName="revoke-keys"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204720    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbfd5177-b27f-4771-8c24-80e86645aa39" containerName="payload-processing"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204725    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbfd5177-b27f-4771-8c24-80e86645aa39" containerName="payload-processing"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204736    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4f039fb-3af1-4197-8f87-f564f47f771a" containerName="payload-processing"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204741    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4f039fb-3af1-4197-8f87-f564f47f771a" containerName="payload-processing"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204748    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05369022-95a6-403e-a1e2-0811151da7ba" containerName="maas-api"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204754    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="05369022-95a6-403e-a1e2-0811151da7ba" containerName="maas-api"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204762    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39533beb-c710-4f71-b669-790299cfb866" containerName="registry-server"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204768    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="39533beb-c710-4f71-b669-790299cfb866" containerName="registry-server"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204813    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a752446f-32cc-47b5-a425-5d4b5d662462" containerName="revoke-keys"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204820    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="66d2afd7-6db6-4fdb-91f9-5735a6b9f0c8" containerName="payload-pre-processing"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204827    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="39533beb-c710-4f71-b669-790299cfb866" containerName="registry-server"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204833    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e13d5469-5380-4b13-97dc-9a5ff72adcd6" containerName="maas-api"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204838    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="05369022-95a6-403e-a1e2-0811151da7ba" containerName="maas-api"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204846    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4f039fb-3af1-4197-8f87-f564f47f771a" containerName="payload-processing"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204852    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4949ccdd-82f7-4a9f-88a7-fb9d96ec5382" containerName="payload-pre-processing"
Jul 26 16:30:00.204951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.204858    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbfd5177-b27f-4771-8c24-80e86645aa39" containerName="payload-processing"
Jul 26 16:30:00.207817 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.207800    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk"
Jul 26 16:30:00.211419 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.211400    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:30:00.216746 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.216729    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk"
Jul 26 16:30:00.229254 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.229235    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk"]
Jul 26 16:30:00.313011 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.312987    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-klmb9_fdc748d3-eddb-4153-b18a-247da49f480f/discovery/0.log"
Jul 26 16:30:00.361212 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.361182    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk"]
Jul 26 16:30:00.367268 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:30:00.367243    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a00c91_341a_4e2a_aaa4_9d4edfebdfc1.slice/crio-7171e137fd91ec6d742a5dd3c05c3fc3262555560f69db4471a77a8da58864aa WatchSource:0}: Error finding container 7171e137fd91ec6d742a5dd3c05c3fc3262555560f69db4471a77a8da58864aa: Status 404 returned error can't find the container with id 7171e137fd91ec6d742a5dd3c05c3fc3262555560f69db4471a77a8da58864aa
Jul 26 16:30:00.421673 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.421651    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-694bf5b6d6-f64vz_3fccad28-d8c4-4e35-b2ee-31aff78e5df3/kube-auth-proxy/0.log"
Jul 26 16:30:00.776002 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.775922    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-64tt4_fff00ece-1226-40dd-bf8d-71c600041701/payload-pre-processing/0.log"
Jul 26 16:30:00.834491 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.834461    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk" event={"ID":"72a00c91-341a-4e2a-aaa4-9d4edfebdfc1","Type":"ContainerStarted","Data":"b8fa84ae7049ea600fcceab138744541d49f642fb5be5f8319999974185244be"}
Jul 26 16:30:00.834817 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.834514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk" event={"ID":"72a00c91-341a-4e2a-aaa4-9d4edfebdfc1","Type":"ContainerStarted","Data":"7171e137fd91ec6d742a5dd3c05c3fc3262555560f69db4471a77a8da58864aa"}
Jul 26 16:30:00.848619 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.848538    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk" podStartSLOduration=0.848521032 podStartE2EDuration="848.521032ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:30:00.848396361 +0000 UTC m=+3116.347786199" watchObservedRunningTime="2026-07-26 16:30:00.848521032 +0000 UTC m=+3116.347910870"
Jul 26 16:30:00.894212 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:00.894191    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-pnh69_f1a3706c-d0ff-4751-8639-a1de8788e821/payload-processing/0.log"
Jul 26 16:30:01.668529 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:01.668492    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rsxn_5c907832-a817-43a9-8284-f91e72c6da3e/main/0.log"
Jul 26 16:30:01.840065 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:01.840019    2560 generic.go:356] "Generic (PLEG): container finished" podID="72a00c91-341a-4e2a-aaa4-9d4edfebdfc1" containerID="b8fa84ae7049ea600fcceab138744541d49f642fb5be5f8319999974185244be" exitCode=0
Jul 26 16:30:01.840549 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:01.840094    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk" event={"ID":"72a00c91-341a-4e2a-aaa4-9d4edfebdfc1","Type":"ContainerDied","Data":"b8fa84ae7049ea600fcceab138744541d49f642fb5be5f8319999974185244be"}
Jul 26 16:30:02.972393 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:02.972371    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk"
Jul 26 16:30:03.850196 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:03.850155    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk" event={"ID":"72a00c91-341a-4e2a-aaa4-9d4edfebdfc1","Type":"ContainerDied","Data":"7171e137fd91ec6d742a5dd3c05c3fc3262555560f69db4471a77a8da58864aa"}
Jul 26 16:30:03.850399 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:03.850210    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7171e137fd91ec6d742a5dd3c05c3fc3262555560f69db4471a77a8da58864aa"
Jul 26 16:30:03.850399 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:03.850166    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751390-4fnkk"
Jul 26 16:30:10.720761 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:10.720731    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4dxwj_9e2d7ff0-73eb-4b22-a8f5-c595d7bb050f/global-pull-secret-syncer/0.log"
Jul 26 16:30:10.944162 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:10.944130    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rx9nc_ff85593b-fd58-4b0b-bfd6-8b378ca48af1/konnectivity-agent/0.log"
Jul 26 16:30:11.020173 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:11.020084    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-10.ec2.internal_660fa30c874fcff3f1c5867b1497b385/haproxy/0.log"
Jul 26 16:30:16.716285 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:16.716243    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cgq5h_dfe51eeb-0d36-455a-a8e7-63e1d4ec1f99/manager/0.log"
Jul 26 16:30:18.819407 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:18.819378    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27c6h_e1ea955f-7f82-42d1-9326-606883aa4bed/node-exporter/0.log"
Jul 26 16:30:18.843415 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:18.843389    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27c6h_e1ea955f-7f82-42d1-9326-606883aa4bed/kube-rbac-proxy/0.log"
Jul 26 16:30:18.867368 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:18.867348    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27c6h_e1ea955f-7f82-42d1-9326-606883aa4bed/init-textfile/0.log"
Jul 26 16:30:20.530550 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.530522    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"]
Jul 26 16:30:20.530939 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.530856    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a00c91-341a-4e2a-aaa4-9d4edfebdfc1" containerName="cleanup"
Jul 26 16:30:20.530939 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.530866    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a00c91-341a-4e2a-aaa4-9d4edfebdfc1" containerName="cleanup"
Jul 26 16:30:20.531027 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.530952    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a00c91-341a-4e2a-aaa4-9d4edfebdfc1" containerName="cleanup"
Jul 26 16:30:20.533836 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.533815    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.536067 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.536042    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4m86b\"/\"default-dockercfg-gbd2s\""
Jul 26 16:30:20.536211 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.536192    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4m86b\"/\"kube-root-ca.crt\""
Jul 26 16:30:20.536689 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.536673    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4m86b\"/\"openshift-service-ca.crt\""
Jul 26 16:30:20.543401 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.543376    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"]
Jul 26 16:30:20.699413 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.699390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-sys\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.699545 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.699420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpms8\" (UniqueName: \"kubernetes.io/projected/a33595d9-408e-4340-9c76-5646cefe7631-kube-api-access-lpms8\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.699545 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.699451    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-proc\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.699545 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.699523    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-podres\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.699545 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.699543    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-lib-modules\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.800050 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.799987    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-sys\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.800050 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.800017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpms8\" (UniqueName: \"kubernetes.io/projected/a33595d9-408e-4340-9c76-5646cefe7631-kube-api-access-lpms8\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.800050 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.800047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-proc\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.800263 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.800072    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-podres\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.800263 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.800093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-lib-modules\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.800263 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.800108    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-sys\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.800263 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.800165    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-proc\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.800263 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.800218    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-podres\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.800263 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.800222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a33595d9-408e-4340-9c76-5646cefe7631-lib-modules\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.808518 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.808503    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpms8\" (UniqueName: \"kubernetes.io/projected/a33595d9-408e-4340-9c76-5646cefe7631-kube-api-access-lpms8\") pod \"perf-node-gather-daemonset-j6lvc\" (UID: \"a33595d9-408e-4340-9c76-5646cefe7631\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.844719 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.844696    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:20.971581 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:20.971557    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"]
Jul 26 16:30:20.973632 ip-10-0-132-10 kubenswrapper[2560]: W0726 16:30:20.973604    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda33595d9_408e_4340_9c76_5646cefe7631.slice/crio-d23cb0188815b453a893ee726183a5b1e2bbf2838fe95911e43376dd89a58033 WatchSource:0}: Error finding container d23cb0188815b453a893ee726183a5b1e2bbf2838fe95911e43376dd89a58033: Status 404 returned error can't find the container with id d23cb0188815b453a893ee726183a5b1e2bbf2838fe95911e43376dd89a58033
Jul 26 16:30:21.928237 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:21.928199    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc" event={"ID":"a33595d9-408e-4340-9c76-5646cefe7631","Type":"ContainerStarted","Data":"495cbba6b0c568e88c117511808c604f4db522d11fa6353dfc1825130dc896af"}
Jul 26 16:30:21.928660 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:21.928251    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc" event={"ID":"a33595d9-408e-4340-9c76-5646cefe7631","Type":"ContainerStarted","Data":"d23cb0188815b453a893ee726183a5b1e2bbf2838fe95911e43376dd89a58033"}
Jul 26 16:30:21.928660 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:21.928343    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:21.951332 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:21.951289    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc" podStartSLOduration=1.9512760340000002 podStartE2EDuration="1.951276034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:30:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:30:21.949792372 +0000 UTC m=+3137.449182210" watchObservedRunningTime="2026-07-26 16:30:21.951276034 +0000 UTC m=+3137.450665872"
Jul 26 16:30:24.002641 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:24.002608    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dssgk_714ceae0-9812-49c6-8a81-5b64ed3bebce/dns/0.log"
Jul 26 16:30:24.023589 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:24.023566    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dssgk_714ceae0-9812-49c6-8a81-5b64ed3bebce/kube-rbac-proxy/0.log"
Jul 26 16:30:24.079672 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:24.079650    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tlcgb_a8c04d11-ba9f-4142-906a-1e10f3ad0954/dns-node-resolver/0.log"
Jul 26 16:30:24.673926 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:24.673882    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7ddc4ddbcf-tqgkr_eebc47af-9e56-4e72-bf4f-7cdf69a1c9d1/registry/0.log"
Jul 26 16:30:24.700596 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:24.700570    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5cr5p_a53b35cf-9867-4dd7-9e53-1cbab673f3fe/node-ca/0.log"
Jul 26 16:30:26.042870 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:26.042840    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-klmb9_fdc748d3-eddb-4153-b18a-247da49f480f/discovery/0.log"
Jul 26 16:30:26.064295 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:26.064268    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-694bf5b6d6-f64vz_3fccad28-d8c4-4e35-b2ee-31aff78e5df3/kube-auth-proxy/0.log"
Jul 26 16:30:26.222907 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:26.222868    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-64tt4_fff00ece-1226-40dd-bf8d-71c600041701/payload-pre-processing/0.log"
Jul 26 16:30:26.268708 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:26.268680    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-pnh69_f1a3706c-d0ff-4751-8639-a1de8788e821/payload-processing/0.log"
Jul 26 16:30:27.152164 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:27.152135    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-28cqg_bcc672a2-bb64-455b-bfd9-8f5ce020365a/serve-healthcheck-canary/0.log"
Jul 26 16:30:27.833986 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:27.833950    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6lcr8_9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663/kube-rbac-proxy/0.log"
Jul 26 16:30:27.858161 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:27.858124    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6lcr8_9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663/exporter/0.log"
Jul 26 16:30:27.882638 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:27.882612    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6lcr8_9e1b35c0-c87e-4b1a-8405-e5c0fb5bb663/extractor/0.log"
Jul 26 16:30:27.941360 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:27.941339    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-j6lvc"
Jul 26 16:30:30.421057 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:30.421023    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qmwhg_49024e42-a310-4da5-b905-cb44535548a0/manager/0.log"
Jul 26 16:30:30.813451 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:30.813362    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sgtxq_a1ef07ad-8d81-4dc9-9f94-160eef929d4f/manager/0.log"
Jul 26 16:30:31.102357 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:31.102264    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86d86cd99-sm5c5_9e01530d-9d12-4e71-b9d1-a311b268f572/manager/0.log"
Jul 26 16:30:31.453231 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:31.453187    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d4685587-6w2v4_a3d0b25c-4d79-4def-99fc-8530a60fc98a/manager/0.log"
Jul 26 16:30:31.474463 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:31.474442    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d4685587-6w2v4_a3d0b25c-4d79-4def-99fc-8530a60fc98a/copy-manifests/0.log"
Jul 26 16:30:31.764215 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:31.764133    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-klmb9_fdc748d3-eddb-4153-b18a-247da49f480f/discovery/0.log"
Jul 26 16:30:31.777530 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:31.777499    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-694bf5b6d6-f64vz_3fccad28-d8c4-4e35-b2ee-31aff78e5df3/kube-auth-proxy/0.log"
Jul 26 16:30:31.914243 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:31.914213    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-64tt4_fff00ece-1226-40dd-bf8d-71c600041701/payload-pre-processing/0.log"
Jul 26 16:30:31.942200 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:31.942164    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-pnh69_f1a3706c-d0ff-4751-8639-a1de8788e821/payload-processing/0.log"
Jul 26 16:30:32.888334 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:32.888302    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-vzrtn_37912504-017f-4ea6-a2df-4b866fb41a29/openshift-lws-operator/0.log"
Jul 26 16:30:33.632212 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:33.632180    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-klmb9_fdc748d3-eddb-4153-b18a-247da49f480f/discovery/0.log"
Jul 26 16:30:33.645738 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:33.645714    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-694bf5b6d6-f64vz_3fccad28-d8c4-4e35-b2ee-31aff78e5df3/kube-auth-proxy/0.log"
Jul 26 16:30:33.783434 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:33.783408    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-64tt4_fff00ece-1226-40dd-bf8d-71c600041701/payload-pre-processing/0.log"
Jul 26 16:30:33.815083 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:33.815054    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-pnh69_f1a3706c-d0ff-4751-8639-a1de8788e821/payload-processing/0.log"
Jul 26 16:30:37.640430 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:37.640401    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-s92qj_eed26f64-6d60-47e4-9086-053d3ccd0b0d/migrator/0.log"
Jul 26 16:30:37.675951 ip-10-0-132-10 kubenswrapper[2560]: I0726 16:30:37.675927    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-s92qj_eed26f64-6d60-47e4-9086-053d3ccd0b0d/graceful-termination/0.log"