Jul 29 00:05:59.157921 ip-10-0-139-127 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 00:05:59.157933 ip-10-0-139-127 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 00:05:59.157941 ip-10-0-139-127 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 00:05:59.158282 ip-10-0-139-127 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 00:06:09.194126 ip-10-0-139-127 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 00:06:09.194146 ip-10-0-139-127 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 5413c260d0694db0af99822467815408 --
Jul 29 00:08:03.416486 ip-10-0-139-127 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 00:08:03.764382 ip-10-0-139-127 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 00:08:03.764382 ip-10-0-139-127 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 00:08:03.764382 ip-10-0-139-127 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 00:08:03.764382 ip-10-0-139-127 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 00:08:03.764382 ip-10-0-139-127 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 00:08:03.766121 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.766040    2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 00:08:03.770032 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770017    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:08:03.770032 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770031    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770035    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770040    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770043    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770047    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770050    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770053    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770056    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770059    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770062    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770064    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770074    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770077    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770080    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770082    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770085    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770088    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770091    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770094    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770097    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:08:03.770094 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770100    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770103    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770106    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770109    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770114    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770118    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770121    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770124    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770127    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770130    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770132    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770135    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770137    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770140    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770143    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770145    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770148    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770150    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770153    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:08:03.770590 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770155    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770158    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770160    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770163    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770165    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770168    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770170    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770173    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770175    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770178    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770180    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770183    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770185    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770187    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770190    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770193    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770196    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770198    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770201    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770204    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:08:03.771071 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770206    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770209    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770212    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770214    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770228    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770230    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770233    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770235    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770238    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770241    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770243    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770246    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770248    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770250    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770253    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770256    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770258    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770261    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770263    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770265    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:08:03.771589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770268    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770270    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770273    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770275    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770279    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770283    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770285    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770291    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770293    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770296    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770299    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770301    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770304    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770308    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770311    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770315    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770318    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770321    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.770323    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:08:03.772089 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771057    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771070    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771074    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771077    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771080    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771083    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771086    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771088    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771091    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771094    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771096    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771099    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771101    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771104    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771114    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771117    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771119    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771122    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771125    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771127    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:08:03.772574 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771131    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771134    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771136    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771139    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771142    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771144    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771147    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771149    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771152    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771161    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771164    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771166    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771169    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771171    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771174    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771176    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771179    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771181    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771184    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771186    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:08:03.773104 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771189    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771191    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771193    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771196    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771199    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771201    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771204    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771206    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771208    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771211    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771213    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771231    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771235    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771237    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771240    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771244    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771248    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771251    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771254    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771257    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:08:03.773647 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771261    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771263    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771272    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771275    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771277    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771281    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771285    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771288    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771290    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771293    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771295    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771298    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771301    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771303    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771306    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771308    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771312    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771316    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771319    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:08:03.774137 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771322    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771326    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771329    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771331    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771334    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771338    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771341    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771344    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771346    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771349    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771351    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771354    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771356    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771359    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771361    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771364    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771367    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771370    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771373    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:08:03.774625 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771375    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771725    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771729    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771732    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771735    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771737    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771740    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771743    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771745    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771748    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771750    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771753    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771755    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771758    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771761    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771763    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771766    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771768    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771771    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771774    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:08:03.775090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771776    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771779    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771782    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771784    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771787    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771790    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771792    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771795    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771799    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771803    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771806    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771810    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771813    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771816    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771819    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771822    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771825    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771827    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771830    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:08:03.775589 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771832    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771835    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771838    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771841    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771843    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771846    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771849    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771852    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771854    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771858    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771861    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771865    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771867    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771870    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771872    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771875    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771877    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771880    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771882    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:08:03.776051 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771885    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771887    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771889    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771892    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771895    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771898    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771900    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771903    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771905    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771908    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771910    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771913    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771915    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771918    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771920    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771923    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771926    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771928    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771931    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771933    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:08:03.776542 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771936    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771939    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771941    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771944    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771947    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771949    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771951    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771954    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771959    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771962    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771964    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771967    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771969    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771972    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771974    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771977    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771979    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771982    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771984    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:08:03.777148 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771987    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771990    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.771992    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772059    2557 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772066    2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772072    2557 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772077    2557 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772080    2557 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772084    2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772087    2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772091    2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772095    2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772098    2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772102    2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772106    2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772109    2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772112    2557 flags.go:64] FLAG: --cgroup-root=""
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772115    2557 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772118    2557 flags.go:64] FLAG: --client-ca-file=""
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772121    2557 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772123    2557 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772129    2557 flags.go:64] FLAG: --cluster-domain=""
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772132    2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772135    2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 00:08:03.777666 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772138    2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772141    2557 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772145    2557 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772148    2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772152    2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772155    2557 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772158    2557 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772161    2557 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772165    2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772168    2557 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772171    2557 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772175    2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772178    2557 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772181    2557 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772184    2557 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772187    2557 flags.go:64] FLAG: --enable-server="true"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772190    2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772194    2557 flags.go:64] FLAG: --event-burst="100"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772197    2557 flags.go:64] FLAG: --event-qps="50"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772200    2557 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772203    2557 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772206    2557 flags.go:64] FLAG: --eviction-hard=""
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772210    2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772213    2557 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772229    2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 00:08:03.778260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772233    2557 flags.go:64] FLAG: --eviction-soft=""
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772236    2557 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772238    2557 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772241    2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772246    2557 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772249    2557 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772252    2557 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772255    2557 flags.go:64] FLAG: --feature-gates=""
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772258    2557 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772262    2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772264    2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772267    2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772270    2557 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772276    2557 flags.go:64] FLAG: --help="false"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772279    2557 flags.go:64] FLAG: --hostname-override="ip-10-0-139-127.ec2.internal"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772283    2557 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772286    2557 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772289    2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772292    2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772296    2557 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772298    2557 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772302    2557 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772304    2557 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 00:08:03.778869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772307    2557 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772310    2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772313    2557 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772316    2557 flags.go:64] FLAG: --kube-reserved=""
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772319    2557 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772322    2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772325    2557 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772327    2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772330    2557 flags.go:64] FLAG: --lock-file=""
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772333    2557 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772336    2557 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772339    2557 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772344    2557 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772349    2557 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772351    2557 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772354    2557 flags.go:64] FLAG: --logging-format="text"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772357    2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772360    2557 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772363    2557 flags.go:64] FLAG: --manifest-url=""
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772366    2557 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772370    2557 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772373    2557 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772383    2557 flags.go:64] FLAG: --max-pods="110"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772386    2557 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772389    2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 00:08:03.779508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772392    2557 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772396    2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772399    2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772402    2557 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772404    2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772411    2557 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772414    2557 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772417    2557 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772421    2557 flags.go:64] FLAG: --pod-cidr=""
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772423    2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772429    2557 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772432    2557 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772435    2557 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772438    2557 flags.go:64] FLAG: --port="10250"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772441    2557 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772443    2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03a19e18611e5fc83"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772446    2557 flags.go:64] FLAG: --qos-reserved=""
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772449    2557 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772453    2557 flags.go:64] FLAG: --register-node="true"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772455    2557 flags.go:64] FLAG: --register-with-taints=""
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772460    2557 flags.go:64] FLAG: --registry-burst="10"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772463    2557 flags.go:64] FLAG: --registry-qps="5"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772466    2557 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772469    2557 flags.go:64] FLAG: --reserved-memory=""
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772472    2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 00:08:03.780109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772475    2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772478    2557 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772481    2557 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772484    2557 flags.go:64] FLAG: --runonce="false"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772487    2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772491    2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772495    2557 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772497    2557 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772501    2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772505    2557 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772508    2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772511    2557 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772513    2557 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772516    2557 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772519    2557 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772522    2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772525    2557 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772528    2557 flags.go:64] FLAG: --system-cgroups=""
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772530    2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772536    2557 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772539    2557 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772541    2557 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772545    2557 flags.go:64] FLAG: --tls-min-version=""
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772548    2557 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772551    2557 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 00:08:03.781036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772554    2557 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772557    2557 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772561    2557 flags.go:64] FLAG: --v="2"
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772565    2557 flags.go:64] FLAG: --version="false"
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772569    2557 flags.go:64] FLAG: --vmodule=""
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772573    2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.772576    2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772687    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772691    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772694    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772697    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772700    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772704    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772707    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772710    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772712    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772715    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772718    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772721    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772723    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772726    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:08:03.781786 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772729    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772731    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772734    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772736    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772739    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772742    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772745    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772747    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772750    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772752    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772755    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772758    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772760    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772764    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772766    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772769    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772771    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772774    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772776    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772779    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:08:03.782340 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772782    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772784    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772788    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772793    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772796    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772799    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772801    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772803    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772807    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772809    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772812    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772815    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772817    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772820    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772822    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772825    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772827    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772830    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772833    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:08:03.782847 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772835    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772838    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772840    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772843    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772845    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772848    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772851    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772854    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772857    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772859    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772862    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772864    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772868    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772872    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772875    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772878    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772882    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772885    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772887    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772890    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:08:03.783359 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772892    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772895    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772898    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772901    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772905    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772908    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772911    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772914    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772917    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772919    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772922    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772925    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772927    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772930    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772932    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772935    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772938    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772940    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772943    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772945    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:08:03.783859 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772948    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772950    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772953    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772955    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772958    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.772960    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.773628    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.781765    2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.781781    2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.782515    2557 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.782524    2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782596    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782601    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782604    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:08:03.784375 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782607    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782610    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782614    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782617    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782620    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782624    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782627    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782630    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782633    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782636    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782639    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782642    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782645    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782648    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782651    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782653    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782656    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782659    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782661    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:08:03.784730 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782664    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782668    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782670    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782673    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782676    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782679    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782682    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782686    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782690    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782693    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782696    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782699    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782702    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782705    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782708    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782711    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782713    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782716    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782719    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782721    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:08:03.785237 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782724    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782727    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782729    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782732    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782734    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782737    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782740    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782743    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782745    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782748    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782750    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782753    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782756    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782759    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782761    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782764    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782767    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782769    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782772    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:08:03.785741 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782775    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782778    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782781    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782783    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782786    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782788    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782791    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782794    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782796    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782800    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782804    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782807    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782810    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782812    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782815    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782818    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782820    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782823    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782826    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782829    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:08:03.786195 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782831    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782834    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782836    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782839    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782842    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782845    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782847    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782850    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782853    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782856    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782858    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782861    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782864    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782868    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782872    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782875    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782878    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:08:03.786784 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782880    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.782885    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.782996    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783000    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783003    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783006    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783009    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783011    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783014    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783017    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783019    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783022    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783024    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783027    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783030    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:08:03.787215 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783033    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783035    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783038    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783040    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783044    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783047    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783049    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783052    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783054    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783057    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783059    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783062    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783065    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783067    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783070    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783072    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783076    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783080    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783083    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:08:03.787608 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783087    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783089    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783092    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783095    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783098    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783100    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783103    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783106    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783109    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783111    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783114    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783117    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783119    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783123    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783125    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783128    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783131    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783133    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783136    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783138    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:08:03.788070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783141    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783143    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783146    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783148    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783151    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783154    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783158    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783161    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783164    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783167    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783169    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783172    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783174    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783177    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783179    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783182    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783184    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783187    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783190    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783192    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:08:03.788566 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783194    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783197    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783200    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783202    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783205    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783207    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783211    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783213    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783231    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783235    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783238    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783240    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783243    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783245    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783248    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783250    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783253    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783256    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783260    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783263    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:08:03.789070 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783266    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783269    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783272    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783275    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783277    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783280    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:03.783282    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.783288    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.783394    2557 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.786591    2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.787754    2557 server.go:1019] "Starting client certificate rotation"
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.787856    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 00:08:03.789583 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.787893    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 00:08:03.789954 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.789941    2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 00:08:03.793946 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.793926    2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 00:08:03.804968 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.804947    2557 log.go:25] "Validated CRI v1 runtime API"
Jul 29 00:08:03.807158 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.807142    2557 log.go:25] "Validated CRI v1 image API"
Jul 29 00:08:03.807941 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.807930    2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 00:08:03.810109 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.810091    2557 fs.go:135] Filesystem UUIDs: map[4be59d25-f4be-4803-9e6f-031c9d45944d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ca8efe81-7791-46d9-8f5f-9e1ef0ee3f73:/dev/nvme0n1p4]
Jul 29 00:08:03.810157 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.810110    2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 00:08:03.815724 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.815609    2557 manager.go:216] Machine: {Timestamp:2026-07-29 00:08:03.814416312 +0000 UTC m=+0.294552117 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3130302 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2dce8aa62d89742914f9d761a6270a SystemUUID:ec2dce8a-a62d-8974-2914-f9d761a6270a BootID:5413c260-d069-4db0-af99-822467815408 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:63:b2:dc:12:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:63:b2:dc:12:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:c0:04:fa:46:25 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 00:08:03.815724 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.815714    2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 00:08:03.815845 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.815789    2557 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 00:08:03.818595 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.818550    2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 00:08:03.819125 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.818700    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 00:08:03.819125 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.818605    2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-127.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 00:08:03.820500 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.820486    2557 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 00:08:03.820570 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.820507    2557 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 00:08:03.820570 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.820519    2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 00:08:03.821428 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.821250    2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 00:08:03.822610 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.822599    2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 00:08:03.826287 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.826275    2557 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 00:08:03.826405 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.826396    2557 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 00:08:03.826445 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.826425    2557 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 00:08:03.826445 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.826435    2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 00:08:03.826516 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.826448    2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 00:08:03.826516 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.826456    2557 kubelet.go:398] "Adding apiserver pod source"
Jul 29 00:08:03.826516 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.826465    2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 00:08:03.826809 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.826798    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 00:08:03.826846 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.826818    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 00:08:03.828788 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.828768    2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 00:08:03.830438 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.830426    2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 00:08:03.830493 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.830455    2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 00:08:03.831037 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831027    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 00:08:03.831071 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831042    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 00:08:03.831071 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831049    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 00:08:03.831071 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831054    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 00:08:03.831071 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831063    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 00:08:03.831071 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831068    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 00:08:03.831200 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831141    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 00:08:03.831200 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831147    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 00:08:03.831200 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831153    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 00:08:03.831200 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831158    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 00:08:03.831200 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831167    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 00:08:03.831200 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831174    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 00:08:03.831862 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831851    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 00:08:03.831862 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.831862    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 00:08:03.835058 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.835044    2557 server.go:1268] "Started kubelet"
Jul 29 00:08:03.835172 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.835120    2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 00:08:03.835258 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.835192    2557 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 00:08:03.835434 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.835406    2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 00:08:03.835902 ip-10-0-139-127 systemd[1]: Started Kubernetes Kubelet.
Jul 29 00:08:03.836711 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.836694    2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 00:08:03.837834 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.837815    2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-127.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 00:08:03.837912 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.837810    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 00:08:03.837979 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.837929    2557 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 00:08:03.838014 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.837970    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-127.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 00:08:03.841846 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.841827    2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 00:08:03.842362 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.842346    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jdmw2"
Jul 29 00:08:03.842866 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.842012    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-127.ec2.internal.18c69a0d3649712d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-127.ec2.internal,UID:ip-10-0-139-127.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-127.ec2.internal,},FirstTimestamp:2026-07-29 00:08:03.835015469 +0000 UTC m=+0.315151274,LastTimestamp:2026-07-29 00:08:03.835015469 +0000 UTC m=+0.315151274,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-127.ec2.internal,}"
Jul 29 00:08:03.843279 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.843266    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 00:08:03.843330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.843269    2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 00:08:03.843407 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.843389    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:03.843891 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.843870    2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 00:08:03.843979 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.843874    2557 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 00:08:03.843979 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.843907    2557 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 00:08:03.844074 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.844019    2557 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 00:08:03.844074 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.844029    2557 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 00:08:03.844239 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.844199    2557 factory.go:153] Registering CRI-O factory
Jul 29 00:08:03.844239 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.844235    2557 factory.go:223] Registration of the crio container factory successfully
Jul 29 00:08:03.844376 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.844334    2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 00:08:03.844376 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.844343    2557 factory.go:55] Registering systemd factory
Jul 29 00:08:03.844376 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.844352    2557 factory.go:223] Registration of the systemd container factory successfully
Jul 29 00:08:03.844376 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.844372    2557 factory.go:103] Registering Raw factory
Jul 29 00:08:03.844518 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.844386    2557 manager.go:1196] Started watching for new ooms in manager
Jul 29 00:08:03.844813 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.844796    2557 manager.go:349] Starting recovery of all containers
Jul 29 00:08:03.847102 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.847066    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 00:08:03.847194 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.847167    2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-127.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 00:08:03.850872 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.850845    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jdmw2"
Jul 29 00:08:03.856663 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.856644    2557 manager.go:354] Recovery completed
Jul 29 00:08:03.858196 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.858162    2557 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 29 00:08:03.861090 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.861078    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:08:03.863363 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.863349    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:08:03.863432 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.863374    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:08:03.863432 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.863386    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:08:03.863884 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.863870    2557 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 00:08:03.863884 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.863877    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:08:03.863884 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.863884    2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 00:08:03.864055 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.863901    2557 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 00:08:03.864542 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.864529    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:08:03.864613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.864557    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:08:03.864613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.864571    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:08:03.865604 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.865589    2557 policy_none.go:49] "None policy: Start"
Jul 29 00:08:03.865604 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.865606    2557 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 00:08:03.865709 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.865618    2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 00:08:03.865709 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.865646    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-127.ec2.internal.18c69a0d37f9f8d5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-127.ec2.internal,UID:ip-10-0-139-127.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-127.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-127.ec2.internal,},FirstTimestamp:2026-07-29 00:08:03.863361749 +0000 UTC m=+0.343497554,LastTimestamp:2026-07-29 00:08:03.863361749 +0000 UTC m=+0.343497554,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-127.ec2.internal,}"
Jul 29 00:08:03.867186 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.867176    2557 policy_none.go:47] "Start"
Jul 29 00:08:03.904859 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.904734    2557 manager.go:337] "Starting Device Plugin manager"
Jul 29 00:08:03.904953 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.904877    2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 00:08:03.904953 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.904892    2557 server.go:87] "Starting device plugin registration server"
Jul 29 00:08:03.905092 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.905081    2557 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 00:08:03.905135 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.905092    2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 00:08:03.905204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.905187    2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 00:08:03.905315 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.905284    2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 00:08:03.905315 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:03.905294    2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 00:08:03.905811 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.905791    2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 00:08:03.905897 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:03.905844    2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:04.005423 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.005399    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:08:04.007169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.006203    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:08:04.007169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.006239    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:08:04.007169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.006251    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:08:04.007169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.006280    2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-127.ec2.internal"
Jul 29 00:08:04.016402 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.016346    2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-127.ec2.internal"
Jul 29 00:08:04.016402 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:04.016368    2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-127.ec2.internal\": node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:04.030965 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:04.030940    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:04.031807 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.031788    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 00:08:04.032981 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.032964    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 00:08:04.033046 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.033011    2557 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 00:08:04.033046 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.033033    2557 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 00:08:04.033132 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:04.033072    2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 00:08:04.035595 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:04.035570    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:08:06.032465 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.032441    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:06.035610 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.035585    2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"]
Jul 29 00:08:06.035677 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.035651    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:08:06.036471 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.036456    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:08:06.036528 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.036490    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:08:06.036528 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.036505    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:08:06.038415 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.038403    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:08:06.038528 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.038513    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.038568 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.038543    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:08:06.039042 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.039026    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:08:06.039042 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.039036    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:08:06.039147 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.039055    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:08:06.039147 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.039070    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:08:06.039147 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.039056    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:08:06.039147 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.039146    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:08:06.041003 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.040986    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.041122 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.041012    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:08:06.041627 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.041613    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:08:06.041723 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.041639    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:08:06.041723 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.041653    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:08:06.046332 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.046315    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeef17ba4dc68f055aaf49917ef0d775-config\") pod \"kube-apiserver-proxy-ip-10-0-139-127.ec2.internal\" (UID: \"aeef17ba4dc68f055aaf49917ef0d775\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.046394 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.046341    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf86ea12642e0c17e83f630b5bf15190-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"cf86ea12642e0c17e83f630b5bf15190\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.046394 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.046359    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf86ea12642e0c17e83f630b5bf15190-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"cf86ea12642e0c17e83f630b5bf15190\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.064304 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.064285    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-127.ec2.internal\" not found" node="ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.068629 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.068613    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-127.ec2.internal\" not found" node="ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.132856 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.132836    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:06.147335 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.147311    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeef17ba4dc68f055aaf49917ef0d775-config\") pod \"kube-apiserver-proxy-ip-10-0-139-127.ec2.internal\" (UID: \"aeef17ba4dc68f055aaf49917ef0d775\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.147428 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.147340    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf86ea12642e0c17e83f630b5bf15190-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"cf86ea12642e0c17e83f630b5bf15190\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.147428 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.147358    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf86ea12642e0c17e83f630b5bf15190-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"cf86ea12642e0c17e83f630b5bf15190\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.147428 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.147401    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf86ea12642e0c17e83f630b5bf15190-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"cf86ea12642e0c17e83f630b5bf15190\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.147428 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.147415    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeef17ba4dc68f055aaf49917ef0d775-config\") pod \"kube-apiserver-proxy-ip-10-0-139-127.ec2.internal\" (UID: \"aeef17ba4dc68f055aaf49917ef0d775\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.147584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.147423    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf86ea12642e0c17e83f630b5bf15190-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"cf86ea12642e0c17e83f630b5bf15190\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.233333 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.233276    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:06.333790 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.333759    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:06.366227 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.366210    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.371946 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.371929    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:06.434532 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.434498    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:06.535026 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.534961    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:06.598166 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.598139    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:08:06.635111 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.635087    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:06.688967 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.688941    2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 00:08:06.689365 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.689110    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 00:08:06.689365 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.689123    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 00:08:06.735474 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.735440    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:06.758475 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.758442    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 00:03:05 +0000 UTC" deadline="2028-01-27 01:54:55.995858007 +0000 UTC"
Jul 29 00:08:06.758475 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.758467    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13129h46m49.237393944s"
Jul 29 00:08:06.835851 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.835760    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:06.898025 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:06.897999    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:08:06.935998 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:06.935971    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:07.027398 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:07.027367    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf86ea12642e0c17e83f630b5bf15190.slice/crio-3127d6ac449bbdbeac03f2a765f8d0f90e1c6dcef35059dadf800d6ad1ce0455 WatchSource:0}: Error finding container 3127d6ac449bbdbeac03f2a765f8d0f90e1c6dcef35059dadf800d6ad1ce0455: Status 404 returned error can't find the container with id 3127d6ac449bbdbeac03f2a765f8d0f90e1c6dcef35059dadf800d6ad1ce0455
Jul 29 00:08:07.027803 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:07.027786    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeef17ba4dc68f055aaf49917ef0d775.slice/crio-1302c31837fc6540ad0af459fb4b9a334df79e298ecb18293d39e4359f66eb5c WatchSource:0}: Error finding container 1302c31837fc6540ad0af459fb4b9a334df79e298ecb18293d39e4359f66eb5c: Status 404 returned error can't find the container with id 1302c31837fc6540ad0af459fb4b9a334df79e298ecb18293d39e4359f66eb5c
Jul 29 00:08:07.032324 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.032307    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:08:07.036980 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.036960    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:07.137629 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.137574    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:07.238307 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.238283    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:07.338772 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.338750    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Jul 29 00:08:07.340667 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.340643    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:08:07.344888 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.344874    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:07.354791 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.354775    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 00:08:07.355732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.355721    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Jul 29 00:08:07.363531 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.363517    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 00:08:07.728645 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.728617    2557 apiserver.go:52] "Watching apiserver"
Jul 29 00:08:07.732871 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.732848    2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 00:08:07.733309 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.733278    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-jbsdb","openshift-network-operator/iptables-alerter-d9tlk","kube-system/konnectivity-agent-cjlpv","openshift-cluster-node-tuning-operator/tuned-4hmm6","openshift-image-registry/node-ca-qz47f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal","openshift-multus/multus-85dnm","openshift-ovn-kubernetes/ovnkube-node-77kdj","kube-system/global-pull-secret-syncer-njnlq","kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279","openshift-dns/node-resolver-j2hc8","openshift-multus/multus-additional-cni-plugins-tsqzq","openshift-multus/network-metrics-daemon-xvzvw"]
Jul 29 00:08:07.735782 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.735763    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.737756 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.737735    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 00:08:07.737756 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.737749    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tg49g\""
Jul 29 00:08:07.737905 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.737779    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 00:08:07.738035 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.738020    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.738085 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.738067    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 00:08:07.738134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.738017    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 00:08:07.739761 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.739600    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gnk4m\""
Jul 29 00:08:07.739761 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.739639    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 00:08:07.739915 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.739784    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 00:08:07.740503 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.740483    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:07.740588 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.740554    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:07.742323 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.742304    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.743841 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.743820    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rrjsf\""
Jul 29 00:08:07.743841 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.743834    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 00:08:07.743996 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.743938    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 00:08:07.744638 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.744622    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:07.745634 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.744935    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 00:08:07.746246 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.746225    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 00:08:07.746354 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.746253    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 00:08:07.746354 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.746341    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 00:08:07.746559 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.746541    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5mvkx\""
Jul 29 00:08:07.747002 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.746984    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.750515 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.749683    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.750515 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.749848    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 00:08:07.750515 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.750182    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 00:08:07.750515 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.750254    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 00:08:07.750515 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.750302    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 00:08:07.750515 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.750396    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 00:08:07.750515 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.750405    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 00:08:07.750901 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.750530    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9b7zq\""
Jul 29 00:08:07.751528 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.751489    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 00:08:07.751528 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.751519    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 00:08:07.751682 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.751601    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 00:08:07.751902 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.751888    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zt4gk\""
Jul 29 00:08:07.752754 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.752733    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:07.754193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754174    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 00:08:07.754297 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754223    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nvch9\""
Jul 29 00:08:07.754416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754395    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-tuned\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.754529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754424    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-run-openvswitch\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.754529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754455    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-log-socket\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.754529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754463    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 00:08:07.754529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754477    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-var-lib-openvswitch\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.754529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754492    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-var-lib-cni-bin\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.754529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754507    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-daemon-config\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.754529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754525    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-sysctl-conf\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754542    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-run-netns\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754578    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eda34408-bf29-4613-acd9-b6db5f690d9f-ovnkube-script-lib\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754597    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754609    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ldqb\" (UniqueName: \"kubernetes.io/projected/eda34408-bf29-4613-acd9-b6db5f690d9f-kube-api-access-6ldqb\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754635    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754674    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-tmp\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754724    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-socket-dir-parent\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754762    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-run-k8s-cni-cncf-io\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754796    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grcf6\" (UniqueName: \"kubernetes.io/projected/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-kube-api-access-grcf6\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754825    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-host\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754852    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-kubelet\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754889    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-run-systemd\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754916    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754940    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-cni-dir\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754962    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk8ws\" (UniqueName: \"kubernetes.io/projected/3201f9ff-9853-4b96-8b6e-0432d625d30c-kube-api-access-zk8ws\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.754980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.754988    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-systemd\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755011    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-sys\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755066    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-slash\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755093    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-cni-binary-copy\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755114    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-kubernetes\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755129    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eda34408-bf29-4613-acd9-b6db5f690d9f-env-overrides\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755149    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-system-cni-dir\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755202    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-var-lib-cni-multus\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755244    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-etc-kubernetes\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755270    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-cnibin\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755294    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755322    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-run-netns\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755343    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-hostroot\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755365    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-os-release\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755425    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755460    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-modprobe-d\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755483    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzfts\" (UniqueName: \"kubernetes.io/projected/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-kube-api-access-wzfts\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.755780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755506    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.755578    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755508    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-system-cni-dir\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755682    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2f59\" (UniqueName: \"kubernetes.io/projected/457be539-8706-406f-b6b0-4df3128fcb60-kube-api-access-v2f59\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755713    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-sysctl-d\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755740    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-var-lib-kubelet\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755764    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-conf-dir\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755785    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-sysconfig\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755805    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7064533e-41f2-46e9-b3ff-a95ae591a5a4-host\") pod \"node-ca-qz47f\" (UID: \"7064533e-41f2-46e9-b3ff-a95ae591a5a4\") " pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755826    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7064533e-41f2-46e9-b3ff-a95ae591a5a4-serviceca\") pod \"node-ca-qz47f\" (UID: \"7064533e-41f2-46e9-b3ff-a95ae591a5a4\") " pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755849    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-cni-bin\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755872    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-lib-modules\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755911    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8mtx\" (UniqueName: \"kubernetes.io/projected/7064533e-41f2-46e9-b3ff-a95ae591a5a4-kube-api-access-q8mtx\") pod \"node-ca-qz47f\" (UID: \"7064533e-41f2-46e9-b3ff-a95ae591a5a4\") " pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755971    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-etc-openvswitch\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.755999    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-var-lib-kubelet\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756040    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-run\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756063    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-node-log\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.756625 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756083    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756087    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-run-multus-certs\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756130    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-run-ovn-kubernetes\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756173    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-cnibin\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756198    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756223    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-systemd-units\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756247    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-run-ovn\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756273    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-cni-netd\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756296    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-os-release\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756332    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3201f9ff-9853-4b96-8b6e-0432d625d30c-cni-binary-copy\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756379    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eda34408-bf29-4613-acd9-b6db5f690d9f-ovnkube-config\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.757339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.756407    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eda34408-bf29-4613-acd9-b6db5f690d9f-ovn-node-metrics-cert\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.757839 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.757820    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:07.757922 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.757901    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:07.760162 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.760141    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:07.761717 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.761610    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 00:08:07.761880 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.761867    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 00:08:07.761971 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.761923    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lrwgv\""
Jul 29 00:08:07.762442 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.762405    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:07.763853 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.763834    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 00:08:07.764114 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.764096    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6dchb\""
Jul 29 00:08:07.764207 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.764102    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 00:08:07.781133 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.781112    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l58sr"
Jul 29 00:08:07.788303 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.788281    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l58sr"
Jul 29 00:08:07.829641 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.829620    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:08:07.847965 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.847372    2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 00:08:07.857023 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.856993    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-var-lib-cni-multus\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.857138 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857036    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-etc-kubernetes\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.857138 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857062    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-cnibin\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.857138 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857086    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.857138 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857115    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9wmf\" (UniqueName: \"kubernetes.io/projected/e9c02508-8e01-440f-bfaf-46d1cf6a4077-kube-api-access-b9wmf\") pod \"iptables-alerter-d9tlk\" (UID: \"e9c02508-8e01-440f-bfaf-46d1cf6a4077\") " pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857139    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-run-netns\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857162    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-hostroot\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857180    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-os-release\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857197    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857214    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-modprobe-d\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857237    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzfts\" (UniqueName: \"kubernetes.io/projected/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-kube-api-access-wzfts\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857252    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-etc-selinux\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857267    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-system-cni-dir\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857295    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2f59\" (UniqueName: \"kubernetes.io/projected/457be539-8706-406f-b6b0-4df3128fcb60-kube-api-access-v2f59\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857318    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-sysctl-d\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.857348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857341    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-var-lib-kubelet\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857363    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-conf-dir\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857417    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-sysconfig\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857443    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7064533e-41f2-46e9-b3ff-a95ae591a5a4-host\") pod \"node-ca-qz47f\" (UID: \"7064533e-41f2-46e9-b3ff-a95ae591a5a4\") " pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857465    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7064533e-41f2-46e9-b3ff-a95ae591a5a4-serviceca\") pod \"node-ca-qz47f\" (UID: \"7064533e-41f2-46e9-b3ff-a95ae591a5a4\") " pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857488    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-cni-bin\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857513    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-kubelet-config\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857538    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-lib-modules\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857561    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8mtx\" (UniqueName: \"kubernetes.io/projected/7064533e-41f2-46e9-b3ff-a95ae591a5a4-kube-api-access-q8mtx\") pod \"node-ca-qz47f\" (UID: \"7064533e-41f2-46e9-b3ff-a95ae591a5a4\") " pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857588    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-etc-openvswitch\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857611    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9c02508-8e01-440f-bfaf-46d1cf6a4077-host-slash\") pod \"iptables-alerter-d9tlk\" (UID: \"e9c02508-8e01-440f-bfaf-46d1cf6a4077\") " pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857635    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-var-lib-kubelet\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857673    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-run\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857699    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-node-log\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857724    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857747    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-socket-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857772    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-run-multus-certs\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.857869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857796    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-run-ovn-kubernetes\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857822    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-cnibin\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857850    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857875    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-systemd-units\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857893    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-run-ovn\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857913    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-cni-netd\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857929    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz9vp\" (UniqueName: \"kubernetes.io/projected/ce867f86-40e1-48ce-9a7d-a3264f1ca054-kube-api-access-tz9vp\") pod \"node-resolver-j2hc8\" (UID: \"ce867f86-40e1-48ce-9a7d-a3264f1ca054\") " pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857945    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-os-release\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857976    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3201f9ff-9853-4b96-8b6e-0432d625d30c-cni-binary-copy\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.857991    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eda34408-bf29-4613-acd9-b6db5f690d9f-ovnkube-config\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858007    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eda34408-bf29-4613-acd9-b6db5f690d9f-ovn-node-metrics-cert\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858023    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcf7h\" (UniqueName: \"kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h\") pod \"network-check-target-jbsdb\" (UID: \"63ef51dc-018c-4a46-83b5-8f056b947beb\") " pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858039    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858055    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/42f50e14-6d95-4eeb-b70c-e03291a251b3-konnectivity-ca\") pod \"konnectivity-agent-cjlpv\" (UID: \"42f50e14-6d95-4eeb-b70c-e03291a251b3\") " pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858069    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-tuned\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858084    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-run-openvswitch\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858101    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-log-socket\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.858746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858115    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce867f86-40e1-48ce-9a7d-a3264f1ca054-tmp-dir\") pod \"node-resolver-j2hc8\" (UID: \"ce867f86-40e1-48ce-9a7d-a3264f1ca054\") " pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858131    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-var-lib-openvswitch\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858145    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-var-lib-cni-bin\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858160    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-daemon-config\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858183    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-sysctl-conf\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858205    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-run-netns\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858227    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eda34408-bf29-4613-acd9-b6db5f690d9f-ovnkube-script-lib\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858243    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ldqb\" (UniqueName: \"kubernetes.io/projected/eda34408-bf29-4613-acd9-b6db5f690d9f-kube-api-access-6ldqb\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858262    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858277    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-tmp\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858291    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-registration-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858306    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ce867f86-40e1-48ce-9a7d-a3264f1ca054-hosts-file\") pod \"node-resolver-j2hc8\" (UID: \"ce867f86-40e1-48ce-9a7d-a3264f1ca054\") " pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858323    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-socket-dir-parent\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858338    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-run-k8s-cni-cncf-io\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858375    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grcf6\" (UniqueName: \"kubernetes.io/projected/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-kube-api-access-grcf6\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858391    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-host\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858406    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-kubelet\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.859514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858425    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-run-systemd\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858451    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858473    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-cni-dir\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858489    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk8ws\" (UniqueName: \"kubernetes.io/projected/3201f9ff-9853-4b96-8b6e-0432d625d30c-kube-api-access-zk8ws\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858504    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-systemd\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858519    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-sys\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858534    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-slash\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858551    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-device-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858567    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-cni-binary-copy\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858583    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-kubernetes\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858607    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eda34408-bf29-4613-acd9-b6db5f690d9f-env-overrides\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858624    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-sys-fs\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858639    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjz62\" (UniqueName: \"kubernetes.io/projected/e5418164-9eac-4499-b10c-c6c02e15bf08-kube-api-access-hjz62\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858815    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-os-release\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858863    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-var-lib-cni-multus\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858904    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-etc-kubernetes\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.858941    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-cnibin\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.860364 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.859283    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3201f9ff-9853-4b96-8b6e-0432d625d30c-cni-binary-copy\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.859415    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.859486    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-run-netns\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.859523    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-hostroot\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.859578    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-os-release\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.859832    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eda34408-bf29-4613-acd9-b6db5f690d9f-ovnkube-config\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.859990    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860111    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-modprobe-d\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860146    2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860390    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-system-cni-dir\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860686    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-sysctl-d\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860741    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-var-lib-kubelet\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860748    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-socket-dir-parent\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860795    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-sysconfig\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860798    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-run-k8s-cni-cncf-io\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860833    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-conf-dir\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860872    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7064533e-41f2-46e9-b3ff-a95ae591a5a4-host\") pod \"node-ca-qz47f\" (UID: \"7064533e-41f2-46e9-b3ff-a95ae591a5a4\") " pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860897    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-cni-binary-copy\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.861195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.860999    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-systemd\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861011    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-host\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861058    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-sys\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861061    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-kubelet\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861099    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-slash\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861105    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-run-systemd\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861148    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861193    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-log-socket\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861203    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-cni-dir\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861233    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-run-openvswitch\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861259    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7064533e-41f2-46e9-b3ff-a95ae591a5a4-serviceca\") pod \"node-ca-qz47f\" (UID: \"7064533e-41f2-46e9-b3ff-a95ae591a5a4\") " pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861306    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-kubernetes\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861309    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-cni-bin\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861407    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-lib-modules\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861446    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-etc-openvswitch\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861453    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-var-lib-cni-bin\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861481    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-var-lib-kubelet\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861494    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-var-lib-openvswitch\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861519    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-run\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861529    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eda34408-bf29-4613-acd9-b6db5f690d9f-env-overrides\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861551    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-node-log\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861584    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-host-run-multus-certs\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861607    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-sysctl-conf\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861617    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-run-ovn-kubernetes\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861677    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-cnibin\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861719    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-cni-netd\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861755    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-systemd-units\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861795    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-run-ovn\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861800    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.861868    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3201f9ff-9853-4b96-8b6e-0432d625d30c-multus-daemon-config\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.861973    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.862039    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs podName:457be539-8706-406f-b6b0-4df3128fcb60 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:08.362020924 +0000 UTC m=+2.940718368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs") pod "network-metrics-daemon-xvzvw" (UID: "457be539-8706-406f-b6b0-4df3128fcb60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.862267    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eda34408-bf29-4613-acd9-b6db5f690d9f-ovnkube-script-lib\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.862584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.862319    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eda34408-bf29-4613-acd9-b6db5f690d9f-host-run-netns\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.863204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.862807    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9c02508-8e01-440f-bfaf-46d1cf6a4077-iptables-alerter-script\") pod \"iptables-alerter-d9tlk\" (UID: \"e9c02508-8e01-440f-bfaf-46d1cf6a4077\") " pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:07.863204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.862851    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-dbus\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:07.863204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.862876    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/42f50e14-6d95-4eeb-b70c-e03291a251b3-agent-certs\") pod \"konnectivity-agent-cjlpv\" (UID: \"42f50e14-6d95-4eeb-b70c-e03291a251b3\") " pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:07.863204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.862905    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-system-cni-dir\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.863204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.862992    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3201f9ff-9853-4b96-8b6e-0432d625d30c-system-cni-dir\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.864564 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.864531    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-tmp\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.865575 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.865496    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eda34408-bf29-4613-acd9-b6db5f690d9f-ovn-node-metrics-cert\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.866171 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.866100    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-etc-tuned\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.869292 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.869241    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2f59\" (UniqueName: \"kubernetes.io/projected/457be539-8706-406f-b6b0-4df3128fcb60-kube-api-access-v2f59\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:07.870217 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.869817    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grcf6\" (UniqueName: \"kubernetes.io/projected/42be31c5-c94c-4fcd-982b-d8cd39f4b38c-kube-api-access-grcf6\") pod \"multus-additional-cni-plugins-tsqzq\" (UID: \"42be31c5-c94c-4fcd-982b-d8cd39f4b38c\") " pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:07.870217 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.870185    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk8ws\" (UniqueName: \"kubernetes.io/projected/3201f9ff-9853-4b96-8b6e-0432d625d30c-kube-api-access-zk8ws\") pod \"multus-85dnm\" (UID: \"3201f9ff-9853-4b96-8b6e-0432d625d30c\") " pod="openshift-multus/multus-85dnm"
Jul 29 00:08:07.871043 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.870992    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzfts\" (UniqueName: \"kubernetes.io/projected/2f44d3e3-2cdd-4140-9d29-e2c0dcb85077-kube-api-access-wzfts\") pod \"tuned-4hmm6\" (UID: \"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077\") " pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:07.871223 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.871195    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8mtx\" (UniqueName: \"kubernetes.io/projected/7064533e-41f2-46e9-b3ff-a95ae591a5a4-kube-api-access-q8mtx\") pod \"node-ca-qz47f\" (UID: \"7064533e-41f2-46e9-b3ff-a95ae591a5a4\") " pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:07.871862 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.871823    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ldqb\" (UniqueName: \"kubernetes.io/projected/eda34408-bf29-4613-acd9-b6db5f690d9f-kube-api-access-6ldqb\") pod \"ovnkube-node-77kdj\" (UID: \"eda34408-bf29-4613-acd9-b6db5f690d9f\") " pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:07.941622 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.941544    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" event={"ID":"cf86ea12642e0c17e83f630b5bf15190","Type":"ContainerStarted","Data":"3127d6ac449bbdbeac03f2a765f8d0f90e1c6dcef35059dadf800d6ad1ce0455"}
Jul 29 00:08:07.944049 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.944023    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" event={"ID":"aeef17ba4dc68f055aaf49917ef0d775","Type":"ContainerStarted","Data":"1302c31837fc6540ad0af459fb4b9a334df79e298ecb18293d39e4359f66eb5c"}
Jul 29 00:08:07.963631 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963604    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcf7h\" (UniqueName: \"kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h\") pod \"network-check-target-jbsdb\" (UID: \"63ef51dc-018c-4a46-83b5-8f056b947beb\") " pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:07.963775 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963642    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:07.963775 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963718    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/42f50e14-6d95-4eeb-b70c-e03291a251b3-konnectivity-ca\") pod \"konnectivity-agent-cjlpv\" (UID: \"42f50e14-6d95-4eeb-b70c-e03291a251b3\") " pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:07.963775 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963758    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce867f86-40e1-48ce-9a7d-a3264f1ca054-tmp-dir\") pod \"node-resolver-j2hc8\" (UID: \"ce867f86-40e1-48ce-9a7d-a3264f1ca054\") " pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:07.963933 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963804    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-registration-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.963933 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963830    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ce867f86-40e1-48ce-9a7d-a3264f1ca054-hosts-file\") pod \"node-resolver-j2hc8\" (UID: \"ce867f86-40e1-48ce-9a7d-a3264f1ca054\") " pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:07.963933 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963866    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-device-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.963933 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963893    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-sys-fs\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.963933 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963918    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjz62\" (UniqueName: \"kubernetes.io/projected/e5418164-9eac-4499-b10c-c6c02e15bf08-kube-api-access-hjz62\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.964207 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963941    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9c02508-8e01-440f-bfaf-46d1cf6a4077-iptables-alerter-script\") pod \"iptables-alerter-d9tlk\" (UID: \"e9c02508-8e01-440f-bfaf-46d1cf6a4077\") " pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:07.964207 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963966    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-dbus\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:07.964207 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.963990    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/42f50e14-6d95-4eeb-b70c-e03291a251b3-agent-certs\") pod \"konnectivity-agent-cjlpv\" (UID: \"42f50e14-6d95-4eeb-b70c-e03291a251b3\") " pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:07.964207 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964019    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9wmf\" (UniqueName: \"kubernetes.io/projected/e9c02508-8e01-440f-bfaf-46d1cf6a4077-kube-api-access-b9wmf\") pod \"iptables-alerter-d9tlk\" (UID: \"e9c02508-8e01-440f-bfaf-46d1cf6a4077\") " pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:07.964207 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.964052    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:07.964207 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.964118    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret podName:a611c2ee-2d8f-4099-ad41-d2be9a0497fa nodeName:}" failed. No retries permitted until 2026-07-29 00:08:08.46409765 +0000 UTC m=+3.042795092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret") pod "global-pull-secret-syncer-njnlq" (UID: "a611c2ee-2d8f-4099-ad41-d2be9a0497fa") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:07.964207 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964168    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-etc-selinux\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.964545 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964053    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-etc-selinux\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.964545 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964330    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-kubelet-config\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:07.964545 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964367    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9c02508-8e01-440f-bfaf-46d1cf6a4077-host-slash\") pod \"iptables-alerter-d9tlk\" (UID: \"e9c02508-8e01-440f-bfaf-46d1cf6a4077\") " pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:07.964545 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964398    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.964545 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964422    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-socket-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.964545 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964455    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz9vp\" (UniqueName: \"kubernetes.io/projected/ce867f86-40e1-48ce-9a7d-a3264f1ca054-kube-api-access-tz9vp\") pod \"node-resolver-j2hc8\" (UID: \"ce867f86-40e1-48ce-9a7d-a3264f1ca054\") " pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964746    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-sys-fs\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964768    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-dbus\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964779    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-kubelet-config\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964811    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/42f50e14-6d95-4eeb-b70c-e03291a251b3-konnectivity-ca\") pod \"konnectivity-agent-cjlpv\" (UID: \"42f50e14-6d95-4eeb-b70c-e03291a251b3\") " pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964832    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964945    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-socket-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964977    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-registration-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964999    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9c02508-8e01-440f-bfaf-46d1cf6a4077-host-slash\") pod \"iptables-alerter-d9tlk\" (UID: \"e9c02508-8e01-440f-bfaf-46d1cf6a4077\") " pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.964999    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5418164-9eac-4499-b10c-c6c02e15bf08-device-dir\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.965014    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce867f86-40e1-48ce-9a7d-a3264f1ca054-tmp-dir\") pod \"node-resolver-j2hc8\" (UID: \"ce867f86-40e1-48ce-9a7d-a3264f1ca054\") " pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:07.965274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.965035    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ce867f86-40e1-48ce-9a7d-a3264f1ca054-hosts-file\") pod \"node-resolver-j2hc8\" (UID: \"ce867f86-40e1-48ce-9a7d-a3264f1ca054\") " pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:07.965799 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.965384    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9c02508-8e01-440f-bfaf-46d1cf6a4077-iptables-alerter-script\") pod \"iptables-alerter-d9tlk\" (UID: \"e9c02508-8e01-440f-bfaf-46d1cf6a4077\") " pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:07.967353 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.967333    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/42f50e14-6d95-4eeb-b70c-e03291a251b3-agent-certs\") pod \"konnectivity-agent-cjlpv\" (UID: \"42f50e14-6d95-4eeb-b70c-e03291a251b3\") " pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:07.971611 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.971589    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:08:07.971611 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.971612    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:08:07.971775 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.971625    2557 projected.go:196] Error preparing data for projected volume kube-api-access-wcf7h for pod openshift-network-diagnostics/network-check-target-jbsdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:07.971775 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:07.971702    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h podName:63ef51dc-018c-4a46-83b5-8f056b947beb nodeName:}" failed. No retries permitted until 2026-07-29 00:08:08.471684105 +0000 UTC m=+3.050381545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wcf7h" (UniqueName: "kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h") pod "network-check-target-jbsdb" (UID: "63ef51dc-018c-4a46-83b5-8f056b947beb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:07.975425 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.975382    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz9vp\" (UniqueName: \"kubernetes.io/projected/ce867f86-40e1-48ce-9a7d-a3264f1ca054-kube-api-access-tz9vp\") pod \"node-resolver-j2hc8\" (UID: \"ce867f86-40e1-48ce-9a7d-a3264f1ca054\") " pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:07.976040 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.976020    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9wmf\" (UniqueName: \"kubernetes.io/projected/e9c02508-8e01-440f-bfaf-46d1cf6a4077-kube-api-access-b9wmf\") pod \"iptables-alerter-d9tlk\" (UID: \"e9c02508-8e01-440f-bfaf-46d1cf6a4077\") " pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:07.977152 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:07.977130    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjz62\" (UniqueName: \"kubernetes.io/projected/e5418164-9eac-4499-b10c-c6c02e15bf08-kube-api-access-hjz62\") pod \"aws-ebs-csi-driver-node-bw279\" (UID: \"e5418164-9eac-4499-b10c-c6c02e15bf08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:08.049098 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.049027    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-85dnm"
Jul 29 00:08:08.056976 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.056814    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tsqzq"
Jul 29 00:08:08.064179 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:08.064148    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3201f9ff_9853_4b96_8b6e_0432d625d30c.slice/crio-e86cb8517c970556a69da10c8aa5504be41ba916c8b723f223190f7687c9673b WatchSource:0}: Error finding container e86cb8517c970556a69da10c8aa5504be41ba916c8b723f223190f7687c9673b: Status 404 returned error can't find the container with id e86cb8517c970556a69da10c8aa5504be41ba916c8b723f223190f7687c9673b
Jul 29 00:08:08.065945 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.065926    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4hmm6"
Jul 29 00:08:08.072129 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.072105    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qz47f"
Jul 29 00:08:08.078879 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.078850    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:08.083532 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:08.083489    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7064533e_41f2_46e9_b3ff_a95ae591a5a4.slice/crio-b5f0d4b61e2bbe0a2a5742a678c22b51323934dfc180e85f63b43c302e31267b WatchSource:0}: Error finding container b5f0d4b61e2bbe0a2a5742a678c22b51323934dfc180e85f63b43c302e31267b: Status 404 returned error can't find the container with id b5f0d4b61e2bbe0a2a5742a678c22b51323934dfc180e85f63b43c302e31267b
Jul 29 00:08:08.086841 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.086679    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279"
Jul 29 00:08:08.091815 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:08.091736    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda34408_bf29_4613_acd9_b6db5f690d9f.slice/crio-df8ee3606192add8c545821b62b99612022bdbdb11f0c93c209bb0f07d7690ce WatchSource:0}: Error finding container df8ee3606192add8c545821b62b99612022bdbdb11f0c93c209bb0f07d7690ce: Status 404 returned error can't find the container with id df8ee3606192add8c545821b62b99612022bdbdb11f0c93c209bb0f07d7690ce
Jul 29 00:08:08.095564 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.095524    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d9tlk"
Jul 29 00:08:08.103178 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:08.103155    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5418164_9eac_4499_b10c_c6c02e15bf08.slice/crio-9db2b0026340de11608d00111ea2238edd46c61d0997baaaf6ad742cd0630e42 WatchSource:0}: Error finding container 9db2b0026340de11608d00111ea2238edd46c61d0997baaaf6ad742cd0630e42: Status 404 returned error can't find the container with id 9db2b0026340de11608d00111ea2238edd46c61d0997baaaf6ad742cd0630e42
Jul 29 00:08:08.104134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.103954    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j2hc8"
Jul 29 00:08:08.107549 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:08.107524    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c02508_8e01_440f_bfaf_46d1cf6a4077.slice/crio-c855ae2fb14d0178e155b36a3b844be07bd6fcd08a68b5f5fdc880db89c97912 WatchSource:0}: Error finding container c855ae2fb14d0178e155b36a3b844be07bd6fcd08a68b5f5fdc880db89c97912: Status 404 returned error can't find the container with id c855ae2fb14d0178e155b36a3b844be07bd6fcd08a68b5f5fdc880db89c97912
Jul 29 00:08:08.112017 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.111998    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:08.114629 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:08.114605    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce867f86_40e1_48ce_9a7d_a3264f1ca054.slice/crio-5a70a605fedec498ac09b3623fbb869788dbad574ba69a573f1f3fc563bb17ff WatchSource:0}: Error finding container 5a70a605fedec498ac09b3623fbb869788dbad574ba69a573f1f3fc563bb17ff: Status 404 returned error can't find the container with id 5a70a605fedec498ac09b3623fbb869788dbad574ba69a573f1f3fc563bb17ff
Jul 29 00:08:08.121965 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:08.121942    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f50e14_6d95_4eeb_b70c_e03291a251b3.slice/crio-158d5e3c865aedd7c8fd59e2dea38290e5c6030d3f82deabc73f8ea974d114dc WatchSource:0}: Error finding container 158d5e3c865aedd7c8fd59e2dea38290e5c6030d3f82deabc73f8ea974d114dc: Status 404 returned error can't find the container with id 158d5e3c865aedd7c8fd59e2dea38290e5c6030d3f82deabc73f8ea974d114dc
Jul 29 00:08:08.366563 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.366477    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:08.366737 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:08.366654    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:08.366737 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:08.366735    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs podName:457be539-8706-406f-b6b0-4df3128fcb60 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:09.366717127 +0000 UTC m=+3.945414565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs") pod "network-metrics-daemon-xvzvw" (UID: "457be539-8706-406f-b6b0-4df3128fcb60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:08.467416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.467377    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:08.467554 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:08.467540    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:08.467623 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:08.467611    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret podName:a611c2ee-2d8f-4099-ad41-d2be9a0497fa nodeName:}" failed. No retries permitted until 2026-07-29 00:08:09.467591182 +0000 UTC m=+4.046288618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret") pod "global-pull-secret-syncer-njnlq" (UID: "a611c2ee-2d8f-4099-ad41-d2be9a0497fa") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:08.568271 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.568242    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcf7h\" (UniqueName: \"kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h\") pod \"network-check-target-jbsdb\" (UID: \"63ef51dc-018c-4a46-83b5-8f056b947beb\") " pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:08.568439 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:08.568371    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:08:08.568439 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:08.568384    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:08:08.568439 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:08.568393    2557 projected.go:196] Error preparing data for projected volume kube-api-access-wcf7h for pod openshift-network-diagnostics/network-check-target-jbsdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:08.568439 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:08.568442    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h podName:63ef51dc-018c-4a46-83b5-8f056b947beb nodeName:}" failed. No retries permitted until 2026-07-29 00:08:09.568427826 +0000 UTC m=+4.147125262 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcf7h" (UniqueName: "kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h") pod "network-check-target-jbsdb" (UID: "63ef51dc-018c-4a46-83b5-8f056b947beb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:08.791348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.791240    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 00:03:07 +0000 UTC" deadline="2028-02-08 12:09:50.28626222 +0000 UTC"
Jul 29 00:08:08.791348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.791295    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13428h1m41.494971814s"
Jul 29 00:08:08.934956 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.934923    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:08.934956 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.934938    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:08.935174 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:08.935053    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:08.935232 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:08.935183    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:08.946521 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.946490    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cjlpv" event={"ID":"42f50e14-6d95-4eeb-b70c-e03291a251b3","Type":"ContainerStarted","Data":"158d5e3c865aedd7c8fd59e2dea38290e5c6030d3f82deabc73f8ea974d114dc"}
Jul 29 00:08:08.947750 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.947707    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j2hc8" event={"ID":"ce867f86-40e1-48ce-9a7d-a3264f1ca054","Type":"ContainerStarted","Data":"5a70a605fedec498ac09b3623fbb869788dbad574ba69a573f1f3fc563bb17ff"}
Jul 29 00:08:08.949008 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.948981    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d9tlk" event={"ID":"e9c02508-8e01-440f-bfaf-46d1cf6a4077","Type":"ContainerStarted","Data":"c855ae2fb14d0178e155b36a3b844be07bd6fcd08a68b5f5fdc880db89c97912"}
Jul 29 00:08:08.950380 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.950299    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" event={"ID":"eda34408-bf29-4613-acd9-b6db5f690d9f","Type":"ContainerStarted","Data":"df8ee3606192add8c545821b62b99612022bdbdb11f0c93c209bb0f07d7690ce"}
Jul 29 00:08:08.951520 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.951497    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qz47f" event={"ID":"7064533e-41f2-46e9-b3ff-a95ae591a5a4","Type":"ContainerStarted","Data":"b5f0d4b61e2bbe0a2a5742a678c22b51323934dfc180e85f63b43c302e31267b"}
Jul 29 00:08:08.952732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.952708    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279" event={"ID":"e5418164-9eac-4499-b10c-c6c02e15bf08","Type":"ContainerStarted","Data":"9db2b0026340de11608d00111ea2238edd46c61d0997baaaf6ad742cd0630e42"}
Jul 29 00:08:08.954016 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.953994    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4hmm6" event={"ID":"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077","Type":"ContainerStarted","Data":"911f11486cfbeee416f101daab21ac8334162d825a4d012e627719c71415273b"}
Jul 29 00:08:08.956340 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.956318    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tsqzq" event={"ID":"42be31c5-c94c-4fcd-982b-d8cd39f4b38c","Type":"ContainerStarted","Data":"c73082c55a17c4fbacbcad13f3754a45b6eaec25faee46c283779b689fd345a4"}
Jul 29 00:08:08.957713 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:08.957692    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-85dnm" event={"ID":"3201f9ff-9853-4b96-8b6e-0432d625d30c","Type":"ContainerStarted","Data":"e86cb8517c970556a69da10c8aa5504be41ba916c8b723f223190f7687c9673b"}
Jul 29 00:08:09.012559 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:09.012532    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:08:09.375363 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:09.375326    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:09.375544 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:09.375473    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:09.375616 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:09.375552    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs podName:457be539-8706-406f-b6b0-4df3128fcb60 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:11.375530757 +0000 UTC m=+5.954228181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs") pod "network-metrics-daemon-xvzvw" (UID: "457be539-8706-406f-b6b0-4df3128fcb60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:09.476511 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:09.476475    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:09.476710 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:09.476617    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:09.476710 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:09.476698    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret podName:a611c2ee-2d8f-4099-ad41-d2be9a0497fa nodeName:}" failed. No retries permitted until 2026-07-29 00:08:11.476678008 +0000 UTC m=+6.055375434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret") pod "global-pull-secret-syncer-njnlq" (UID: "a611c2ee-2d8f-4099-ad41-d2be9a0497fa") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:09.577554 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:09.577516    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcf7h\" (UniqueName: \"kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h\") pod \"network-check-target-jbsdb\" (UID: \"63ef51dc-018c-4a46-83b5-8f056b947beb\") " pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:09.577750 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:09.577736    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:08:09.577815 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:09.577758    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:08:09.577815 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:09.577773    2557 projected.go:196] Error preparing data for projected volume kube-api-access-wcf7h for pod openshift-network-diagnostics/network-check-target-jbsdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:09.577922 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:09.577830    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h podName:63ef51dc-018c-4a46-83b5-8f056b947beb nodeName:}" failed. No retries permitted until 2026-07-29 00:08:11.577813009 +0000 UTC m=+6.156510434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcf7h" (UniqueName: "kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h") pod "network-check-target-jbsdb" (UID: "63ef51dc-018c-4a46-83b5-8f056b947beb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:09.791904 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:09.791867    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 00:03:07 +0000 UTC" deadline="2028-01-22 02:54:15.034339357 +0000 UTC"
Jul 29 00:08:09.792365 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:09.791907    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13010h46m5.242438591s"
Jul 29 00:08:09.937568 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:09.937287    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:09.937732 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:09.937609    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:09.976265 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:09.976229    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" event={"ID":"aeef17ba4dc68f055aaf49917ef0d775","Type":"ContainerStarted","Data":"6a0abc5d6b340c093c3628eaae7187567731b1f83864e4bdebb860fd456aba43"}
Jul 29 00:08:09.989021 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:09.988963    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" podStartSLOduration=2.98894396 podStartE2EDuration="2.98894396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:09.988155439 +0000 UTC m=+4.566852885" watchObservedRunningTime="2026-07-29 00:08:09.98894396 +0000 UTC m=+4.567641407"
Jul 29 00:08:10.935424 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:10.935396    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:10.935945 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:10.935406    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:10.935945 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:10.935546    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:10.935945 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:10.935598    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:10.999244 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:10.999210    2557 generic.go:356] "Generic (PLEG): container finished" podID="cf86ea12642e0c17e83f630b5bf15190" containerID="3de51740748c6d561e83164fd5b572408908647d1630bdfca7034b61c0cddd34" exitCode=0
Jul 29 00:08:10.999415 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:10.999316    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" event={"ID":"cf86ea12642e0c17e83f630b5bf15190","Type":"ContainerDied","Data":"3de51740748c6d561e83164fd5b572408908647d1630bdfca7034b61c0cddd34"}
Jul 29 00:08:11.393214 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:11.392510    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:11.393214 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:11.392719    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:11.393214 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:11.392790    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs podName:457be539-8706-406f-b6b0-4df3128fcb60 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:15.392769574 +0000 UTC m=+9.971467021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs") pod "network-metrics-daemon-xvzvw" (UID: "457be539-8706-406f-b6b0-4df3128fcb60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:11.494307 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:11.493746    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:11.494307 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:11.493898    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:11.494307 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:11.493961    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret podName:a611c2ee-2d8f-4099-ad41-d2be9a0497fa nodeName:}" failed. No retries permitted until 2026-07-29 00:08:15.493943239 +0000 UTC m=+10.072640691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret") pod "global-pull-secret-syncer-njnlq" (UID: "a611c2ee-2d8f-4099-ad41-d2be9a0497fa") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:11.595471 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:11.594785    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcf7h\" (UniqueName: \"kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h\") pod \"network-check-target-jbsdb\" (UID: \"63ef51dc-018c-4a46-83b5-8f056b947beb\") " pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:11.595471 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:11.595006    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:08:11.595471 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:11.595028    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:08:11.595471 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:11.595052    2557 projected.go:196] Error preparing data for projected volume kube-api-access-wcf7h for pod openshift-network-diagnostics/network-check-target-jbsdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:11.595471 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:11.595113    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h podName:63ef51dc-018c-4a46-83b5-8f056b947beb nodeName:}" failed. No retries permitted until 2026-07-29 00:08:15.595094316 +0000 UTC m=+10.173791743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcf7h" (UniqueName: "kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h") pod "network-check-target-jbsdb" (UID: "63ef51dc-018c-4a46-83b5-8f056b947beb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:11.937966 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:11.937477    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:11.937966 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:11.937609    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:12.935634 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:12.935600    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:12.935832 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:12.935762    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:12.935895 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:12.935840    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:12.935942 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:12.935906    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:13.935756 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:13.935714    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:13.936162 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:13.935858    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:14.935226 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:14.935193    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:14.935226 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:14.935218    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:14.935488 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:14.935321    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:14.935759 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:14.935706    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:15.426308 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:15.426266    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:15.426784 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:15.426428    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:15.426784 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:15.426506    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs podName:457be539-8706-406f-b6b0-4df3128fcb60 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:23.426486863 +0000 UTC m=+18.005184292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs") pod "network-metrics-daemon-xvzvw" (UID: "457be539-8706-406f-b6b0-4df3128fcb60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:15.527568 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:15.527332    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:15.527568 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:15.527503    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:15.527568 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:15.527572    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret podName:a611c2ee-2d8f-4099-ad41-d2be9a0497fa nodeName:}" failed. No retries permitted until 2026-07-29 00:08:23.527554982 +0000 UTC m=+18.106252405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret") pod "global-pull-secret-syncer-njnlq" (UID: "a611c2ee-2d8f-4099-ad41-d2be9a0497fa") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:15.628513 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:15.628479    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcf7h\" (UniqueName: \"kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h\") pod \"network-check-target-jbsdb\" (UID: \"63ef51dc-018c-4a46-83b5-8f056b947beb\") " pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:15.628726 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:15.628625    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:08:15.628726 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:15.628641    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:08:15.628726 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:15.628649    2557 projected.go:196] Error preparing data for projected volume kube-api-access-wcf7h for pod openshift-network-diagnostics/network-check-target-jbsdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:15.628726 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:15.628717    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h podName:63ef51dc-018c-4a46-83b5-8f056b947beb nodeName:}" failed. No retries permitted until 2026-07-29 00:08:23.628699258 +0000 UTC m=+18.207396684 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcf7h" (UniqueName: "kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h") pod "network-check-target-jbsdb" (UID: "63ef51dc-018c-4a46-83b5-8f056b947beb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:15.936174 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:15.935675    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:15.936174 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:15.935805    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:16.934739 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:16.934712    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:16.935190 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:16.934712    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:16.935190 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:16.934827    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:16.935190 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:16.934898    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:17.935521 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:17.935431    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:17.936002 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:17.935565    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:18.935460 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:18.935424    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:18.935653 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:18.935476    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:18.935653 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:18.935576    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:18.936062 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:18.935730    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:19.935186 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:19.934981    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:19.935186 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:19.935137    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:20.935285 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:20.935251    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:20.935285 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:20.935283    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:20.935790 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:20.935384    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:20.935790 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:20.935515    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:21.938946 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:21.938917    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:21.939355 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:21.939046    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:22.935271 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:22.935242    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:22.935481 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:22.935242    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:22.935481 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:22.935350    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:22.935481 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:22.935437    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:23.484073 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:23.484034    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:23.484625 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:23.484187    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:23.484625 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:23.484262    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs podName:457be539-8706-406f-b6b0-4df3128fcb60 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:39.484242243 +0000 UTC m=+34.062939682 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs") pod "network-metrics-daemon-xvzvw" (UID: "457be539-8706-406f-b6b0-4df3128fcb60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:23.584340 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:23.584309    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:23.584519 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:23.584458    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:23.584569 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:23.584523    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret podName:a611c2ee-2d8f-4099-ad41-d2be9a0497fa nodeName:}" failed. No retries permitted until 2026-07-29 00:08:39.584507686 +0000 UTC m=+34.163205109 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret") pod "global-pull-secret-syncer-njnlq" (UID: "a611c2ee-2d8f-4099-ad41-d2be9a0497fa") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:23.685613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:23.685571    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcf7h\" (UniqueName: \"kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h\") pod \"network-check-target-jbsdb\" (UID: \"63ef51dc-018c-4a46-83b5-8f056b947beb\") " pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:23.685821 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:23.685779    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:08:23.685821 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:23.685805    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:08:23.685821 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:23.685819    2557 projected.go:196] Error preparing data for projected volume kube-api-access-wcf7h for pod openshift-network-diagnostics/network-check-target-jbsdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:23.685957 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:23.685883    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h podName:63ef51dc-018c-4a46-83b5-8f056b947beb nodeName:}" failed. No retries permitted until 2026-07-29 00:08:39.685867938 +0000 UTC m=+34.264565374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcf7h" (UniqueName: "kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h") pod "network-check-target-jbsdb" (UID: "63ef51dc-018c-4a46-83b5-8f056b947beb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:23.935758 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:23.935729    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:23.935945 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:23.935853    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:24.934915 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:24.934876    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:24.935347 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:24.934887    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:24.935347 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:24.935009    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:24.935347 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:24.935122    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:25.936514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:25.936478    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:25.936999 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:25.936596    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:26.934814 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:26.934782    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:26.935008 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:26.934786    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:26.935008 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:26.934906    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:26.935119 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:26.935010    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:27.935918 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:27.935725    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:27.936716 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:27.936029    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:28.034312 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.034231    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4hmm6" event={"ID":"2f44d3e3-2cdd-4140-9d29-e2c0dcb85077","Type":"ContainerStarted","Data":"dc3f000c8d6614fd07b974deb69ebd71e8dfd744f91c48739680880a6d02c951"}
Jul 29 00:08:28.035782 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.035754    2557 generic.go:356] "Generic (PLEG): container finished" podID="42be31c5-c94c-4fcd-982b-d8cd39f4b38c" containerID="1bf1bace48935ff3e5c4a7aca38afa2d7534ccf36b990ffb890dfd937ade8274" exitCode=0
Jul 29 00:08:28.035893 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.035823    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tsqzq" event={"ID":"42be31c5-c94c-4fcd-982b-d8cd39f4b38c","Type":"ContainerDied","Data":"1bf1bace48935ff3e5c4a7aca38afa2d7534ccf36b990ffb890dfd937ade8274"}
Jul 29 00:08:28.037416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.037391    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-85dnm" event={"ID":"3201f9ff-9853-4b96-8b6e-0432d625d30c","Type":"ContainerStarted","Data":"4e7a37c41eec2102fed62c07de2772a177b596b6561b1ba326a2beec7626d064"}
Jul 29 00:08:28.039376 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.039354    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" event={"ID":"cf86ea12642e0c17e83f630b5bf15190","Type":"ContainerStarted","Data":"7f5f6e37a0793a4cb30e91fd6a2de47cd4c312229247e52a9f2fa7a0971c3c85"}
Jul 29 00:08:28.040684 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.040646    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cjlpv" event={"ID":"42f50e14-6d95-4eeb-b70c-e03291a251b3","Type":"ContainerStarted","Data":"1f45eca38dc2f9ea9981b058b463c1b3d5c60302743b133263c9968aa625481b"}
Jul 29 00:08:28.041954 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.041935    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j2hc8" event={"ID":"ce867f86-40e1-48ce-9a7d-a3264f1ca054","Type":"ContainerStarted","Data":"acc5c9020d86b178e60fd6621f9ec872b542394cfa7ed5194e59bb4df373f10b"}
Jul 29 00:08:28.044587 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.044571    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:08:28.044920 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.044904    2557 generic.go:356] "Generic (PLEG): container finished" podID="eda34408-bf29-4613-acd9-b6db5f690d9f" containerID="d43adc0c3dd4acaf91dcd4182f49528d389e353a7c2a9903a02d2af9bd992c4d" exitCode=1
Jul 29 00:08:28.044995 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.044960    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" event={"ID":"eda34408-bf29-4613-acd9-b6db5f690d9f","Type":"ContainerStarted","Data":"01ae30116dcfa93ac36183b4b38e7b0dd7662279f1da652fc6a2d6ea85e14564"}
Jul 29 00:08:28.045051 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.045002    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" event={"ID":"eda34408-bf29-4613-acd9-b6db5f690d9f","Type":"ContainerStarted","Data":"f8e965ce8a62a8c9a2ab54e977172970fe308f1f817c8a8afda9cf9493e60379"}
Jul 29 00:08:28.045051 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.045043    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" event={"ID":"eda34408-bf29-4613-acd9-b6db5f690d9f","Type":"ContainerStarted","Data":"b479882629ac4b23b5a4e70f4dc151770e180870bbc34b6fd96ea545725e7450"}
Jul 29 00:08:28.045128 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.045083    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" event={"ID":"eda34408-bf29-4613-acd9-b6db5f690d9f","Type":"ContainerStarted","Data":"aa0b52108cb0e66fd3f6adde621383eda4149c6560bb5daf9aa14faeed6f843f"}
Jul 29 00:08:28.045128 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.045113    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" event={"ID":"eda34408-bf29-4613-acd9-b6db5f690d9f","Type":"ContainerDied","Data":"d43adc0c3dd4acaf91dcd4182f49528d389e353a7c2a9903a02d2af9bd992c4d"}
Jul 29 00:08:28.045206 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.045151    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" event={"ID":"eda34408-bf29-4613-acd9-b6db5f690d9f","Type":"ContainerStarted","Data":"33816ec3679cfa86883bc2c4d985a33021d220214a8adb22e33684414d743c76"}
Jul 29 00:08:28.046185 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.046166    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qz47f" event={"ID":"7064533e-41f2-46e9-b3ff-a95ae591a5a4","Type":"ContainerStarted","Data":"b8d4b0dd71b57f45940278ae6fa49eaca9f4c1d596fddc3bef2736dc3011b6a2"}
Jul 29 00:08:28.047443 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.047423    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279" event={"ID":"e5418164-9eac-4499-b10c-c6c02e15bf08","Type":"ContainerStarted","Data":"da63895f13658a742acc5d20363d4c92ec5493230935d50e48d80dd6acc7a66b"}
Jul 29 00:08:28.050127 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.050091    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4hmm6" podStartSLOduration=3.980716707 podStartE2EDuration="23.050080749s" totalImagesPullingTime="19.069364042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:28.049906099 +0000 UTC m=+22.628603544" watchObservedRunningTime="2026-07-29 00:08:28.050080749 +0000 UTC m=+22.628778194"
Jul 29 00:08:28.061932 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.061903    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" podStartSLOduration=21.061897074 podStartE2EDuration="21.061897074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:28.061765636 +0000 UTC m=+22.640463080" watchObservedRunningTime="2026-07-29 00:08:28.061897074 +0000 UTC m=+22.640594519"
Jul 29 00:08:28.095855 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.095810    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-85dnm" podStartSLOduration=3.493663211 podStartE2EDuration="23.095793507s" totalImagesPullingTime="19.602130296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:28.095162831 +0000 UTC m=+22.673860277" watchObservedRunningTime="2026-07-29 00:08:28.095793507 +0000 UTC m=+22.674490954"
Jul 29 00:08:28.108559 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.108530    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qz47f" podStartSLOduration=4.04447756 podStartE2EDuration="23.108519931s" totalImagesPullingTime="19.064042371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:28.108288205 +0000 UTC m=+22.686985650" watchObservedRunningTime="2026-07-29 00:08:28.108519931 +0000 UTC m=+22.687217376"
Jul 29 00:08:28.124452 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.124412    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j2hc8" podStartSLOduration=3.092032303 podStartE2EDuration="22.124398595s" totalImagesPullingTime="19.032366292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:28.124380461 +0000 UTC m=+22.703077907" watchObservedRunningTime="2026-07-29 00:08:28.124398595 +0000 UTC m=+22.703096040"
Jul 29 00:08:28.138497 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.138465    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cjlpv" podStartSLOduration=7.283819872 podStartE2EDuration="22.138454768s" totalImagesPullingTime="14.854634896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:08:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:28.138274264 +0000 UTC m=+22.716971708" watchObservedRunningTime="2026-07-29 00:08:28.138454768 +0000 UTC m=+22.717152211"
Jul 29 00:08:28.935289 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.935263    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:28.935479 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:28.935359    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:28.935479 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:28.935390    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:28.935479 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:28.935450    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:29.051597 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:29.051565    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d9tlk" event={"ID":"e9c02508-8e01-440f-bfaf-46d1cf6a4077","Type":"ContainerStarted","Data":"249c26f14e08800beca71d40d14aff366aed70d70be27186ba9c21deeb05a02e"}
Jul 29 00:08:29.064547 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:29.064506    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d9tlk" podStartSLOduration=5.025493847 podStartE2EDuration="24.064488013s" totalImagesPullingTime="19.038994166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:29.064123437 +0000 UTC m=+23.642820882" watchObservedRunningTime="2026-07-29 00:08:29.064488013 +0000 UTC m=+23.643185458"
Jul 29 00:08:29.086517 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:29.086487    2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 00:08:29.819333 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:29.819194    2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T00:08:29.086515057Z","UUID":"7efea5b9-461a-4be1-b90b-5b574653478c","Handler":null,"Name":"","Endpoint":""}
Jul 29 00:08:29.820771 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:29.820750    2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 00:08:29.820911 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:29.820790    2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 00:08:29.935192 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:29.935165    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:29.935333 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:29.935303    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:30.055961 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:30.055923    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279" event={"ID":"e5418164-9eac-4499-b10c-c6c02e15bf08","Type":"ContainerStarted","Data":"2f58fe39dc385f042ae33576bdbaab6badda88c5bb411572a3aa1a0dde79d25a"}
Jul 29 00:08:30.302347 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:30.302325    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:30.302691 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:30.302649    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:30.935349 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:30.935317    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:30.935498 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:30.935317    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:30.935498 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:30.935440    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:30.935498 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:30.935480    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:31.062265 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:31.062102    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:08:31.062713 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:31.062687    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" event={"ID":"eda34408-bf29-4613-acd9-b6db5f690d9f","Type":"ContainerStarted","Data":"4178585f09113c32c2325d079cc52e76e71cc9dd2a4afd6faf82f21776f8d23a"}
Jul 29 00:08:31.065014 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:31.064983    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279" event={"ID":"e5418164-9eac-4499-b10c-c6c02e15bf08","Type":"ContainerStarted","Data":"b9908a1b68005a7b09dfe33e3eee79c6b395307b6de9817d36f17cc788adb6e5"}
Jul 29 00:08:31.065408 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:31.065386    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:31.065820 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:31.065800    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cjlpv"
Jul 29 00:08:31.081806 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:31.081767    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bw279" podStartSLOduration=4.282065687 podStartE2EDuration="26.081755387s" totalImagesPullingTime="21.7996897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:31.081670441 +0000 UTC m=+25.660367882" watchObservedRunningTime="2026-07-29 00:08:31.081755387 +0000 UTC m=+25.660452831"
Jul 29 00:08:31.934786 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:31.934749    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:31.934971 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:31.934884    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:32.935169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:32.934975    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:32.935845 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:32.935025    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:32.935845 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:32.935257    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:32.935845 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:32.935311    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:33.072631 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:33.072610    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:08:33.073011 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:33.072980    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" event={"ID":"eda34408-bf29-4613-acd9-b6db5f690d9f","Type":"ContainerStarted","Data":"b57357bf63f23021eec6bda39e8501041bf998d389ae41eeab7539fd44da6ef9"}
Jul 29 00:08:33.073304 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:33.073282    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:33.073406 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:33.073337    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:33.073531 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:33.073507    2557 scope.go:117] "RemoveContainer" containerID="d43adc0c3dd4acaf91dcd4182f49528d389e353a7c2a9903a02d2af9bd992c4d"
Jul 29 00:08:33.074846 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:33.074814    2557 generic.go:356] "Generic (PLEG): container finished" podID="42be31c5-c94c-4fcd-982b-d8cd39f4b38c" containerID="4554e35a70dbfc1fc5e1b69e2dc2aa508fa3fa4b8ef3d20b4b5a5675e4afc5d0" exitCode=0
Jul 29 00:08:33.074964 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:33.074873    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tsqzq" event={"ID":"42be31c5-c94c-4fcd-982b-d8cd39f4b38c","Type":"ContainerDied","Data":"4554e35a70dbfc1fc5e1b69e2dc2aa508fa3fa4b8ef3d20b4b5a5675e4afc5d0"}
Jul 29 00:08:33.088820 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:33.088756    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:33.935168 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:33.935131    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:33.935711 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:33.935256    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:34.081392 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.081325    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:08:34.081705 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.081679    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" event={"ID":"eda34408-bf29-4613-acd9-b6db5f690d9f","Type":"ContainerStarted","Data":"b162f5f871c2e6e72f466e6226c15487e546bf6a5114db971e5d4f8e9b937feb"}
Jul 29 00:08:34.081948 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.081928    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:34.097818 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.097796    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:08:34.105846 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.105799    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj" podStartSLOduration=9.999380506 podStartE2EDuration="29.105779839s" totalImagesPullingTime="19.106399333s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:34.104495126 +0000 UTC m=+28.683192613" watchObservedRunningTime="2026-07-29 00:08:34.105779839 +0000 UTC m=+28.684477285"
Jul 29 00:08:34.660713 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.660464    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-njnlq"]
Jul 29 00:08:34.660909 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.660791    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:34.660909 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:34.660891    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:34.663580 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.663553    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jbsdb"]
Jul 29 00:08:34.663751 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.663690    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:34.663819 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:34.663791    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:34.664278 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.664246    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xvzvw"]
Jul 29 00:08:34.664384 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:34.664361    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:34.664466 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:34.664448    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:35.085793 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:35.085716    2557 generic.go:356] "Generic (PLEG): container finished" podID="42be31c5-c94c-4fcd-982b-d8cd39f4b38c" containerID="b1464e42817943415eac3ae8e586da85ecd1d5f25719643dad32aa314c65df9b" exitCode=0
Jul 29 00:08:35.085793 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:35.085779    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tsqzq" event={"ID":"42be31c5-c94c-4fcd-982b-d8cd39f4b38c","Type":"ContainerDied","Data":"b1464e42817943415eac3ae8e586da85ecd1d5f25719643dad32aa314c65df9b"}
Jul 29 00:08:36.935696 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:36.935666    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:36.936247 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:36.935671    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:36.936247 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:36.935766    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:36.936247 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:36.935862    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:36.936247 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:36.935670    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:36.936247 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:36.935948    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:37.092406 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:37.092324    2557 generic.go:356] "Generic (PLEG): container finished" podID="42be31c5-c94c-4fcd-982b-d8cd39f4b38c" containerID="f636ffef41f2ca3587e359bd8add7cf88fcad3780a962e822077088aec96d9fd" exitCode=0
Jul 29 00:08:37.092406 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:37.092380    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tsqzq" event={"ID":"42be31c5-c94c-4fcd-982b-d8cd39f4b38c","Type":"ContainerDied","Data":"f636ffef41f2ca3587e359bd8add7cf88fcad3780a962e822077088aec96d9fd"}
Jul 29 00:08:38.935281 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:38.935245    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:38.935281 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:38.935273    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:38.935811 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:38.935259    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:38.935811 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:38.935382    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvzvw" podUID="457be539-8706-406f-b6b0-4df3128fcb60"
Jul 29 00:08:38.935811 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:38.935513    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbsdb" podUID="63ef51dc-018c-4a46-83b5-8f056b947beb"
Jul 29 00:08:38.935811 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:38.935582    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-njnlq" podUID="a611c2ee-2d8f-4099-ad41-d2be9a0497fa"
Jul 29 00:08:39.504853 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:39.504820    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:39.505034 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:39.504949    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:39.505034 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:39.505016    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs podName:457be539-8706-406f-b6b0-4df3128fcb60 nodeName:}" failed. No retries permitted until 2026-07-29 00:09:11.504998334 +0000 UTC m=+66.083695759 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs") pod "network-metrics-daemon-xvzvw" (UID: "457be539-8706-406f-b6b0-4df3128fcb60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:08:39.605927 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:39.605887    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:39.606180 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:39.606060    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:39.606180 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:39.606134    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret podName:a611c2ee-2d8f-4099-ad41-d2be9a0497fa nodeName:}" failed. No retries permitted until 2026-07-29 00:09:11.606115587 +0000 UTC m=+66.184813026 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret") pod "global-pull-secret-syncer-njnlq" (UID: "a611c2ee-2d8f-4099-ad41-d2be9a0497fa") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:08:39.706525 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:39.706486    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcf7h\" (UniqueName: \"kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h\") pod \"network-check-target-jbsdb\" (UID: \"63ef51dc-018c-4a46-83b5-8f056b947beb\") " pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:39.706710 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:39.706628    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:08:39.706710 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:39.706670    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:08:39.706710 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:39.706685    2557 projected.go:196] Error preparing data for projected volume kube-api-access-wcf7h for pod openshift-network-diagnostics/network-check-target-jbsdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:39.706852 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:39.706743    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h podName:63ef51dc-018c-4a46-83b5-8f056b947beb nodeName:}" failed. No retries permitted until 2026-07-29 00:09:11.706725997 +0000 UTC m=+66.285423443 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wcf7h" (UniqueName: "kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h") pod "network-check-target-jbsdb" (UID: "63ef51dc-018c-4a46-83b5-8f056b947beb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:08:40.255618 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.255585    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeReady"
Jul 29 00:08:40.256164 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.255731    2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 00:08:40.287071 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.287045    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"]
Jul 29 00:08:40.291367 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.291344    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.292900 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.292871    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 00:08:40.293023 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.292980    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 00:08:40.293089 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.293032    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 00:08:40.293177 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.293159    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qq8mm\""
Jul 29 00:08:40.295072 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.295050    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tm4dr"]
Jul 29 00:08:40.298235 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.298211    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.302758 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.299976    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ptczj"]
Jul 29 00:08:40.302758 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.300327    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 00:08:40.302758 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.301585    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wvlt2\""
Jul 29 00:08:40.302758 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.301689    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 00:08:40.302758 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.301844    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 00:08:40.304983 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.303783    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ptczj"
Jul 29 00:08:40.304983 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.304336    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"]
Jul 29 00:08:40.305774 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.305433    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 00:08:40.305774 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.305543    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jmk87\""
Jul 29 00:08:40.305774 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.305647    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 00:08:40.305774 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.305672    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 00:08:40.310195 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.310173    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tm4dr"]
Jul 29 00:08:40.310915 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.310847    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ptczj"]
Jul 29 00:08:40.412914 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.412881    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-installation-pull-secrets\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.412914 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.412917    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.413134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.412946    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-bound-sa-token\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.413134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413015    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bff27cb-47c4-412d-8e1f-8de040279666-tmp-dir\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.413134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413054    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7bl4\" (UniqueName: \"kubernetes.io/projected/4bff27cb-47c4-412d-8e1f-8de040279666-kube-api-access-j7bl4\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.413134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413082    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zfbt\" (UniqueName: \"kubernetes.io/projected/c9af53b3-10d5-4075-acd0-421f63b08456-kube-api-access-9zfbt\") pod \"ingress-canary-ptczj\" (UID: \"c9af53b3-10d5-4075-acd0-421f63b08456\") " pod="openshift-ingress-canary/ingress-canary-ptczj"
Jul 29 00:08:40.413339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413199    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-certificates\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.413339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413235    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-trusted-ca\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.413339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413298    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4bff27cb-47c4-412d-8e1f-8de040279666-config-volume\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.413339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413328    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9af53b3-10d5-4075-acd0-421f63b08456-cert\") pod \"ingress-canary-ptczj\" (UID: \"c9af53b3-10d5-4075-acd0-421f63b08456\") " pod="openshift-ingress-canary/ingress-canary-ptczj"
Jul 29 00:08:40.413529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413364    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-image-registry-private-configuration\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.413529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413391    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.413529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413439    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-ca-trust-extracted\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.413529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.413471    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgr9k\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-kube-api-access-tgr9k\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.514166 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514134    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4bff27cb-47c4-412d-8e1f-8de040279666-config-volume\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.514166 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514170    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9af53b3-10d5-4075-acd0-421f63b08456-cert\") pod \"ingress-canary-ptczj\" (UID: \"c9af53b3-10d5-4075-acd0-421f63b08456\") " pod="openshift-ingress-canary/ingress-canary-ptczj"
Jul 29 00:08:40.514402 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514196    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-image-registry-private-configuration\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.514402 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514217    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.514402 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514250    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-ca-trust-extracted\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.514402 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:40.514328    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 00:08:40.514402 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:40.514349    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq: secret "image-registry-tls" not found
Jul 29 00:08:40.514617 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:40.514421    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls podName:06fb17c2-a242-45bf-9d86-3cb89f2b9a9b nodeName:}" failed. No retries permitted until 2026-07-29 00:08:41.014399977 +0000 UTC m=+35.593097414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls") pod "image-registry-6cb7fdbf5d-p6gxq" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b") : secret "image-registry-tls" not found
Jul 29 00:08:40.514617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514557    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgr9k\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-kube-api-access-tgr9k\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.514741 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514615    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-installation-pull-secrets\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.514741 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514637    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.514741 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514680    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-bound-sa-token\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.514741 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514704    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bff27cb-47c4-412d-8e1f-8de040279666-tmp-dir\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.514741 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514725    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7bl4\" (UniqueName: \"kubernetes.io/projected/4bff27cb-47c4-412d-8e1f-8de040279666-kube-api-access-j7bl4\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.514973 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514745    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zfbt\" (UniqueName: \"kubernetes.io/projected/c9af53b3-10d5-4075-acd0-421f63b08456-kube-api-access-9zfbt\") pod \"ingress-canary-ptczj\" (UID: \"c9af53b3-10d5-4075-acd0-421f63b08456\") " pod="openshift-ingress-canary/ingress-canary-ptczj"
Jul 29 00:08:40.514973 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514768    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-ca-trust-extracted\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.514973 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514796    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-certificates\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.514973 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514818    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-trusted-ca\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.514973 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.514831    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4bff27cb-47c4-412d-8e1f-8de040279666-config-volume\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.514973 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:40.514953    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:08:40.515214 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:40.515018    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls podName:4bff27cb-47c4-412d-8e1f-8de040279666 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:41.014997855 +0000 UTC m=+35.593695295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls") pod "dns-default-tm4dr" (UID: "4bff27cb-47c4-412d-8e1f-8de040279666") : secret "dns-default-metrics-tls" not found
Jul 29 00:08:40.515214 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.515118    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bff27cb-47c4-412d-8e1f-8de040279666-tmp-dir\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.515523 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.515489    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-certificates\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.515973 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.515953    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-trusted-ca\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.518770 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.518751    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-installation-pull-secrets\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.518880 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.518772    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-image-registry-private-configuration\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.518880 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.518809    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9af53b3-10d5-4075-acd0-421f63b08456-cert\") pod \"ingress-canary-ptczj\" (UID: \"c9af53b3-10d5-4075-acd0-421f63b08456\") " pod="openshift-ingress-canary/ingress-canary-ptczj"
Jul 29 00:08:40.523182 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.523161    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7bl4\" (UniqueName: \"kubernetes.io/projected/4bff27cb-47c4-412d-8e1f-8de040279666-kube-api-access-j7bl4\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:40.523816 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.523793    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-bound-sa-token\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.524014 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.523989    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgr9k\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-kube-api-access-tgr9k\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:40.524425 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.524403    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zfbt\" (UniqueName: \"kubernetes.io/projected/c9af53b3-10d5-4075-acd0-421f63b08456-kube-api-access-9zfbt\") pod \"ingress-canary-ptczj\" (UID: \"c9af53b3-10d5-4075-acd0-421f63b08456\") " pod="openshift-ingress-canary/ingress-canary-ptczj"
Jul 29 00:08:40.622687 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.622636    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ptczj"
Jul 29 00:08:40.935557 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.935524    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:08:40.935744 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.935527    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:08:40.935744 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.935677    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:08:40.937727 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.937705    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 00:08:40.937858 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.937753    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 00:08:40.937947 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.937911    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 00:08:40.938027 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.937979    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 00:08:40.938084 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.938073    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-thdv6\""
Jul 29 00:08:40.938138 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:40.938097    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pdg8h\""
Jul 29 00:08:41.019009 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:41.018978    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:41.019147 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:41.019033    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:41.019147 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:41.019128    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 00:08:41.019147 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:41.019146    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq: secret "image-registry-tls" not found
Jul 29 00:08:41.019337 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:41.019165    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:08:41.019337 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:41.019203    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls podName:06fb17c2-a242-45bf-9d86-3cb89f2b9a9b nodeName:}" failed. No retries permitted until 2026-07-29 00:08:42.019185341 +0000 UTC m=+36.597882765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls") pod "image-registry-6cb7fdbf5d-p6gxq" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b") : secret "image-registry-tls" not found
Jul 29 00:08:41.019337 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:41.019227    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls podName:4bff27cb-47c4-412d-8e1f-8de040279666 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:42.019211357 +0000 UTC m=+36.597908803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls") pod "dns-default-tm4dr" (UID: "4bff27cb-47c4-412d-8e1f-8de040279666") : secret "dns-default-metrics-tls" not found
Jul 29 00:08:41.129176 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:41.129150    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ptczj"]
Jul 29 00:08:41.134309 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:41.134288    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9af53b3_10d5_4075_acd0_421f63b08456.slice/crio-e7187a446f4ec1cb57049ff12717f7bd24e7a9c8ef705b1426060cbb81958492 WatchSource:0}: Error finding container e7187a446f4ec1cb57049ff12717f7bd24e7a9c8ef705b1426060cbb81958492: Status 404 returned error can't find the container with id e7187a446f4ec1cb57049ff12717f7bd24e7a9c8ef705b1426060cbb81958492
Jul 29 00:08:42.026803 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.026766    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:42.027277 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.026875    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:42.027277 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:42.026918    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:08:42.027277 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:42.026989    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls podName:4bff27cb-47c4-412d-8e1f-8de040279666 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:44.026973117 +0000 UTC m=+38.605670562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls") pod "dns-default-tm4dr" (UID: "4bff27cb-47c4-412d-8e1f-8de040279666") : secret "dns-default-metrics-tls" not found
Jul 29 00:08:42.027277 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:42.027072    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 00:08:42.027277 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:42.027090    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq: secret "image-registry-tls" not found
Jul 29 00:08:42.027277 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:42.027151    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls podName:06fb17c2-a242-45bf-9d86-3cb89f2b9a9b nodeName:}" failed. No retries permitted until 2026-07-29 00:08:44.027134956 +0000 UTC m=+38.605832386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls") pod "image-registry-6cb7fdbf5d-p6gxq" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b") : secret "image-registry-tls" not found
Jul 29 00:08:42.109095 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.109057    2557 generic.go:356] "Generic (PLEG): container finished" podID="42be31c5-c94c-4fcd-982b-d8cd39f4b38c" containerID="7ee2a564b219d367adfcb3443d7ad0120f04fd13a80ad5846a940f7dce3420d8" exitCode=0
Jul 29 00:08:42.109257 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.109109    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tsqzq" event={"ID":"42be31c5-c94c-4fcd-982b-d8cd39f4b38c","Type":"ContainerDied","Data":"7ee2a564b219d367adfcb3443d7ad0120f04fd13a80ad5846a940f7dce3420d8"}
Jul 29 00:08:42.110610 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.110583    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ptczj" event={"ID":"c9af53b3-10d5-4075-acd0-421f63b08456","Type":"ContainerStarted","Data":"e7187a446f4ec1cb57049ff12717f7bd24e7a9c8ef705b1426060cbb81958492"}
Jul 29 00:08:42.839765 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.839535    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vd2sr"]
Jul 29 00:08:42.844358 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.844336    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:42.848911 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:42.848875    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"insights-runtime-extractor-vd2sr\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="e94e1019-75e2-4403-bfa0-20e6fdc8bd51" pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:42.848911 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:42.848893    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 29 00:08:42.854020 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.853858    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 00:08:42.854020 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.853886    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 00:08:42.854020 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.853898    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 00:08:42.854243 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.854165    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gmgkv\""
Jul 29 00:08:42.884861 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.884829    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vd2sr"]
Jul 29 00:08:42.933943 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.933907    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-crio-socket\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:42.933943 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.933950    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:42.934159 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.933986    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-data-volume\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:42.934159 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.934116    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:42.934252 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.934174    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klpf5\" (UniqueName: \"kubernetes.io/projected/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-api-access-klpf5\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:42.963915 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.963888    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg"]
Jul 29 00:08:42.967017 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.966994    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg"
Jul 29 00:08:42.970861 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.970839    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 29 00:08:42.970987 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.970867    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 29 00:08:42.970987 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.970872    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bsm2t\""
Jul 29 00:08:42.982255 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:42.982205    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg"]
Jul 29 00:08:43.035202 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.035172    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-crio-socket\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:43.035613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.035207    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:43.035613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.035235    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-data-volume\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:43.035613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.035299    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:43.035613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.035343    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klpf5\" (UniqueName: \"kubernetes.io/projected/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-api-access-klpf5\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:43.035613 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:43.035362    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:43.035613 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:43.035413    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:08:43.035613 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:43.035427    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:43.535408798 +0000 UTC m=+38.114106238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:43.035613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.035456    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-crio-socket\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:43.035613 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:43.035484    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:43.535464737 +0000 UTC m=+38.114162179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : configmap "kube-rbac-proxy" not found
Jul 29 00:08:43.035613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.035613    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-data-volume\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:43.116984 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.116951    2557 generic.go:356] "Generic (PLEG): container finished" podID="42be31c5-c94c-4fcd-982b-d8cd39f4b38c" containerID="7a748101fc9c6acbfc403015da6b2a23cac402e46b739bfbf472179863d9a640" exitCode=0
Jul 29 00:08:43.117128 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.117024    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tsqzq" event={"ID":"42be31c5-c94c-4fcd-982b-d8cd39f4b38c","Type":"ContainerDied","Data":"7a748101fc9c6acbfc403015da6b2a23cac402e46b739bfbf472179863d9a640"}
Jul 29 00:08:43.136373 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.136346    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdlc8\" (UniqueName: \"kubernetes.io/projected/683ea5f6-3cec-4bc4-b979-9ea0385646b8-kube-api-access-gdlc8\") pod \"migrator-6865c9967-dhmhg\" (UID: \"683ea5f6-3cec-4bc4-b979-9ea0385646b8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg"
Jul 29 00:08:43.237391 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.237038    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdlc8\" (UniqueName: \"kubernetes.io/projected/683ea5f6-3cec-4bc4-b979-9ea0385646b8-kube-api-access-gdlc8\") pod \"migrator-6865c9967-dhmhg\" (UID: \"683ea5f6-3cec-4bc4-b979-9ea0385646b8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg"
Jul 29 00:08:43.246703 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.246675    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdlc8\" (UniqueName: \"kubernetes.io/projected/683ea5f6-3cec-4bc4-b979-9ea0385646b8-kube-api-access-gdlc8\") pod \"migrator-6865c9967-dhmhg\" (UID: \"683ea5f6-3cec-4bc4-b979-9ea0385646b8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg"
Jul 29 00:08:43.277218 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.277182    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg"
Jul 29 00:08:43.539860 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.539777    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:43.539860 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.539860    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:43.540030 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:43.539929    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:43.540030 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:43.539953    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:08:43.540030 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:43.539996    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:44.539981502 +0000 UTC m=+39.118678925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:43.540030 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:43.540012    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:44.540003785 +0000 UTC m=+39.118701208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : configmap "kube-rbac-proxy" not found
Jul 29 00:08:43.700022 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:43.699998    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg"]
Jul 29 00:08:43.705238 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:43.705206    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683ea5f6_3cec_4bc4_b979_9ea0385646b8.slice/crio-e19c315134d346fe10fecf68ed27df3caa61d88f92ca166f85c73722c64cfba8 WatchSource:0}: Error finding container e19c315134d346fe10fecf68ed27df3caa61d88f92ca166f85c73722c64cfba8: Status 404 returned error can't find the container with id e19c315134d346fe10fecf68ed27df3caa61d88f92ca166f85c73722c64cfba8
Jul 29 00:08:44.042139 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.041965    2557 projected.go:291] Couldn't get configMap openshift-insights/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 29 00:08:44.042139 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.042144    2557 projected.go:196] Error preparing data for projected volume kube-api-access-klpf5 for pod openshift-insights/insights-runtime-extractor-vd2sr: failed to sync configmap cache: timed out waiting for the condition
Jul 29 00:08:44.042841 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.042205    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-api-access-klpf5 podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:44.542187947 +0000 UTC m=+39.120885383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-klpf5" (UniqueName: "kubernetes.io/projected/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-api-access-klpf5") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : failed to sync configmap cache: timed out waiting for the condition
Jul 29 00:08:44.044333 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.044306    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:44.044442 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.044376    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:44.044483 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.044448    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 00:08:44.044483 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.044464    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq: secret "image-registry-tls" not found
Jul 29 00:08:44.044552 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.044507    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:08:44.044590 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.044510    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls podName:06fb17c2-a242-45bf-9d86-3cb89f2b9a9b nodeName:}" failed. No retries permitted until 2026-07-29 00:08:48.044498194 +0000 UTC m=+42.623195617 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls") pod "image-registry-6cb7fdbf5d-p6gxq" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b") : secret "image-registry-tls" not found
Jul 29 00:08:44.044590 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.044571    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls podName:4bff27cb-47c4-412d-8e1f-8de040279666 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:48.044558796 +0000 UTC m=+42.623256219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls") pod "dns-default-tm4dr" (UID: "4bff27cb-47c4-412d-8e1f-8de040279666") : secret "dns-default-metrics-tls" not found
Jul 29 00:08:44.123094 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.123064    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tsqzq" event={"ID":"42be31c5-c94c-4fcd-982b-d8cd39f4b38c","Type":"ContainerStarted","Data":"edbce8920249d27bdac59a7b44b175320b055b05b9d2ead68fe225eccc92d02e"}
Jul 29 00:08:44.124506 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.124477    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ptczj" event={"ID":"c9af53b3-10d5-4075-acd0-421f63b08456","Type":"ContainerStarted","Data":"c2723e7e81507a928b0ac4863aee0ad26e40c5b77c327e8012c5e10fdfdf579e"}
Jul 29 00:08:44.125504 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.125486    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg" event={"ID":"683ea5f6-3cec-4bc4-b979-9ea0385646b8","Type":"ContainerStarted","Data":"e19c315134d346fe10fecf68ed27df3caa61d88f92ca166f85c73722c64cfba8"}
Jul 29 00:08:44.145961 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.145924    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tsqzq" podStartSLOduration=8.886171197 podStartE2EDuration="39.145912688s" totalImagesPullingTime="30.259741491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:44.14457139 +0000 UTC m=+38.723268835" watchObservedRunningTime="2026-07-29 00:08:44.145912688 +0000 UTC m=+38.724610132"
Jul 29 00:08:44.159718 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.159679    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ptczj" podStartSLOduration=1.507572283 podStartE2EDuration="4.159651594s" totalImagesPullingTime="2.652079311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:44.159126473 +0000 UTC m=+38.737823921" watchObservedRunningTime="2026-07-29 00:08:44.159651594 +0000 UTC m=+38.738349040"
Jul 29 00:08:44.289644 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.289617    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 00:08:44.393260 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.393191    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j2hc8_ce867f86-40e1-48ce-9a7d-a3264f1ca054/dns-node-resolver/0.log"
Jul 29 00:08:44.548653 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.548619    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:44.548821 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.548679    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klpf5\" (UniqueName: \"kubernetes.io/projected/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-api-access-klpf5\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:44.548821 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.548738    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:08:44.548821 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.548757    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:44.548821 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.548806    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:46.548788924 +0000 UTC m=+41.127486369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : configmap "kube-rbac-proxy" not found
Jul 29 00:08:44.548968 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.548842    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:44.548968 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:44.548893    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:46.548879747 +0000 UTC m=+41.127577183 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:44.551893 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:44.551872    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klpf5\" (UniqueName: \"kubernetes.io/projected/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-api-access-klpf5\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:45.592351 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.592321    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qz47f_7064533e-41f2-46e9-b3ff-a95ae591a5a4/node-ca/0.log"
Jul 29 00:08:45.726679 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.726631    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-z9sfk"]
Jul 29 00:08:45.742901 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.742868    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-z9sfk"]
Jul 29 00:08:45.743056 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.742960    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:45.744950 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.744924    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 29 00:08:45.745227 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.745211    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 29 00:08:45.745227 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.745220    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 29 00:08:45.745425 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.745231    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 29 00:08:45.745425 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.745221    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-kksmc\""
Jul 29 00:08:45.859081 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.859055    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvztw\" (UniqueName: \"kubernetes.io/projected/d3ca2bf5-058c-4e11-9065-bf57694dd32c-kube-api-access-nvztw\") pod \"service-ca-658d7b7c78-z9sfk\" (UID: \"d3ca2bf5-058c-4e11-9065-bf57694dd32c\") " pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:45.859212 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.859188    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d3ca2bf5-058c-4e11-9065-bf57694dd32c-signing-key\") pod \"service-ca-658d7b7c78-z9sfk\" (UID: \"d3ca2bf5-058c-4e11-9065-bf57694dd32c\") " pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:45.859273 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.859229    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d3ca2bf5-058c-4e11-9065-bf57694dd32c-signing-cabundle\") pod \"service-ca-658d7b7c78-z9sfk\" (UID: \"d3ca2bf5-058c-4e11-9065-bf57694dd32c\") " pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:45.959686 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.959645    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvztw\" (UniqueName: \"kubernetes.io/projected/d3ca2bf5-058c-4e11-9065-bf57694dd32c-kube-api-access-nvztw\") pod \"service-ca-658d7b7c78-z9sfk\" (UID: \"d3ca2bf5-058c-4e11-9065-bf57694dd32c\") " pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:45.959814 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.959800    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d3ca2bf5-058c-4e11-9065-bf57694dd32c-signing-key\") pod \"service-ca-658d7b7c78-z9sfk\" (UID: \"d3ca2bf5-058c-4e11-9065-bf57694dd32c\") " pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:45.959883 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.959829    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d3ca2bf5-058c-4e11-9065-bf57694dd32c-signing-cabundle\") pod \"service-ca-658d7b7c78-z9sfk\" (UID: \"d3ca2bf5-058c-4e11-9065-bf57694dd32c\") " pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:45.960410 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.960388    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d3ca2bf5-058c-4e11-9065-bf57694dd32c-signing-cabundle\") pod \"service-ca-658d7b7c78-z9sfk\" (UID: \"d3ca2bf5-058c-4e11-9065-bf57694dd32c\") " pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:45.961974 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.961953    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d3ca2bf5-058c-4e11-9065-bf57694dd32c-signing-key\") pod \"service-ca-658d7b7c78-z9sfk\" (UID: \"d3ca2bf5-058c-4e11-9065-bf57694dd32c\") " pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:45.966750 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:45.966724    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvztw\" (UniqueName: \"kubernetes.io/projected/d3ca2bf5-058c-4e11-9065-bf57694dd32c-kube-api-access-nvztw\") pod \"service-ca-658d7b7c78-z9sfk\" (UID: \"d3ca2bf5-058c-4e11-9065-bf57694dd32c\") " pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:46.053094 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:46.053071    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk"
Jul 29 00:08:46.133505 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:46.133477    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg" event={"ID":"683ea5f6-3cec-4bc4-b979-9ea0385646b8","Type":"ContainerStarted","Data":"945cee6ad8c3903b2e199ad71ac515445ce937734879ad9983a7a3937dcac4a9"}
Jul 29 00:08:46.133614 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:46.133536    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg" event={"ID":"683ea5f6-3cec-4bc4-b979-9ea0385646b8","Type":"ContainerStarted","Data":"a73452cf07143cce2a5d7d87df7ffddfe8a0dc187692fa6c272b2ed0e4dce46c"}
Jul 29 00:08:46.147429 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:46.147385    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-dhmhg" podStartSLOduration=1.957901284 podStartE2EDuration="4.147370007s" totalImagesPullingTime="2.189468723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:46.146285979 +0000 UTC m=+40.724983425" watchObservedRunningTime="2026-07-29 00:08:46.147370007 +0000 UTC m=+40.726067455"
Jul 29 00:08:46.193624 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:46.193596    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ptczj_c9af53b3-10d5-4075-acd0-421f63b08456/serve-healthcheck-canary/0.log"
Jul 29 00:08:46.203049 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:46.203023    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-z9sfk"]
Jul 29 00:08:46.206966 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:46.206944    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ca2bf5_058c_4e11_9065_bf57694dd32c.slice/crio-18ada077891f9f1dd87efa3b2db31d08697474dfc534c48ee9ab933d517bfed4 WatchSource:0}: Error finding container 18ada077891f9f1dd87efa3b2db31d08697474dfc534c48ee9ab933d517bfed4: Status 404 returned error can't find the container with id 18ada077891f9f1dd87efa3b2db31d08697474dfc534c48ee9ab933d517bfed4
Jul 29 00:08:46.563566 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:46.563471    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:46.563746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:46.563567    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:46.563746 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:46.563630    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:46.563746 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:46.563700    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:08:46.563746 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:46.563717    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:50.563701848 +0000 UTC m=+45.142399276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:46.563909 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:46.563750    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:50.563734793 +0000 UTC m=+45.142432246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : configmap "kube-rbac-proxy" not found
Jul 29 00:08:47.137201 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:47.137165    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk" event={"ID":"d3ca2bf5-058c-4e11-9065-bf57694dd32c","Type":"ContainerStarted","Data":"18ada077891f9f1dd87efa3b2db31d08697474dfc534c48ee9ab933d517bfed4"}
Jul 29 00:08:48.077067 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:48.077028    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:48.077235 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:48.077098    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:48.077291 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:48.077241    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:08:48.077291 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:48.077259    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 00:08:48.077291 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:48.077274    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq: secret "image-registry-tls" not found
Jul 29 00:08:48.077433 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:48.077319    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls podName:06fb17c2-a242-45bf-9d86-3cb89f2b9a9b nodeName:}" failed. No retries permitted until 2026-07-29 00:08:56.077304688 +0000 UTC m=+50.656002125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls") pod "image-registry-6cb7fdbf5d-p6gxq" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b") : secret "image-registry-tls" not found
Jul 29 00:08:48.077433 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:48.077335    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls podName:4bff27cb-47c4-412d-8e1f-8de040279666 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:56.077326564 +0000 UTC m=+50.656023992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls") pod "dns-default-tm4dr" (UID: "4bff27cb-47c4-412d-8e1f-8de040279666") : secret "dns-default-metrics-tls" not found
Jul 29 00:08:49.143979 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:49.143945    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk" event={"ID":"d3ca2bf5-058c-4e11-9065-bf57694dd32c","Type":"ContainerStarted","Data":"f64bb67a6e9fdccd31b58ad5ce97fad544d110520f84b8cf745923f4894ebd34"}
Jul 29 00:08:49.158743 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:49.158684    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-z9sfk" podStartSLOduration=1.6579448810000001 podStartE2EDuration="4.158642268s" totalImagesPullingTime="2.500697387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:49.158572325 +0000 UTC m=+43.737269771" watchObservedRunningTime="2026-07-29 00:08:49.158642268 +0000 UTC m=+43.737339715"
Jul 29 00:08:50.593912 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:50.593863    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:50.594503 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:50.593962    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:50.594503 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:50.594070    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:08:50.594503 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:50.594131    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:58.594112714 +0000 UTC m=+53.172810145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : configmap "kube-rbac-proxy" not found
Jul 29 00:08:50.594690 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:50.594529    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:50.594690 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:50.594576    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:08:58.594562719 +0000 UTC m=+53.173260149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:56.136407 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:56.136366    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:56.136897 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:56.136496    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:56.138904 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:56.138811    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4bff27cb-47c4-412d-8e1f-8de040279666-metrics-tls\") pod \"dns-default-tm4dr\" (UID: \"4bff27cb-47c4-412d-8e1f-8de040279666\") " pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:56.139053 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:56.138956    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls\") pod \"image-registry-6cb7fdbf5d-p6gxq\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") " pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:56.207830 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:56.207802    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:56.215892 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:56.215866    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:56.370786 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:56.370710    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"]
Jul 29 00:08:56.375289 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:56.375267    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tm4dr"]
Jul 29 00:08:56.375400 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:56.375271    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06fb17c2_a242_45bf_9d86_3cb89f2b9a9b.slice/crio-649a209d80a586c944d6ef12dbfdb266e12ce11cd252902ef29cd66ba728199c WatchSource:0}: Error finding container 649a209d80a586c944d6ef12dbfdb266e12ce11cd252902ef29cd66ba728199c: Status 404 returned error can't find the container with id 649a209d80a586c944d6ef12dbfdb266e12ce11cd252902ef29cd66ba728199c
Jul 29 00:08:56.380719 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:08:56.380696    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bff27cb_47c4_412d_8e1f_8de040279666.slice/crio-b5d89493737e5f0cb1abd64ccddeac147bee2ad0384e4620f8ffea1f88d37db1 WatchSource:0}: Error finding container b5d89493737e5f0cb1abd64ccddeac147bee2ad0384e4620f8ffea1f88d37db1: Status 404 returned error can't find the container with id b5d89493737e5f0cb1abd64ccddeac147bee2ad0384e4620f8ffea1f88d37db1
Jul 29 00:08:57.183593 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:57.183547    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tm4dr" event={"ID":"4bff27cb-47c4-412d-8e1f-8de040279666","Type":"ContainerStarted","Data":"b5d89493737e5f0cb1abd64ccddeac147bee2ad0384e4620f8ffea1f88d37db1"}
Jul 29 00:08:57.185085 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:57.185058    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq" event={"ID":"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b","Type":"ContainerStarted","Data":"177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7"}
Jul 29 00:08:57.185219 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:57.185097    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq" event={"ID":"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b","Type":"ContainerStarted","Data":"649a209d80a586c944d6ef12dbfdb266e12ce11cd252902ef29cd66ba728199c"}
Jul 29 00:08:57.185306 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:57.185287    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:08:57.202053 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:57.201899    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq" podStartSLOduration=23.20185209 podStartE2EDuration="23.20185209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:08:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:57.20121879 +0000 UTC m=+51.779916236" watchObservedRunningTime="2026-07-29 00:08:57.20185209 +0000 UTC m=+51.780549534"
Jul 29 00:08:58.656048 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:58.656012    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:58.656405 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:58.656079    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:58.656405 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:58.656183    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:58.656405 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:08:58.656254    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls podName:e94e1019-75e2-4403-bfa0-20e6fdc8bd51 nodeName:}" failed. No retries permitted until 2026-07-29 00:09:14.656234471 +0000 UTC m=+69.234931914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vd2sr" (UID: "e94e1019-75e2-4403-bfa0-20e6fdc8bd51") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:08:58.656611 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:58.656588    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:08:59.194217 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:59.194132    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tm4dr" event={"ID":"4bff27cb-47c4-412d-8e1f-8de040279666","Type":"ContainerStarted","Data":"7d8c67873e18a6d6f9a2c0d8d071e51e459c5d4cd8ce3ffd33afe41bc9ea0a07"}
Jul 29 00:08:59.194217 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:59.194184    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tm4dr" event={"ID":"4bff27cb-47c4-412d-8e1f-8de040279666","Type":"ContainerStarted","Data":"4e88a790c5a398e05692871017358802b491b55519da706e8e0e2e0514bd7c2f"}
Jul 29 00:08:59.194449 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:59.194227    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:08:59.208622 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:08:59.208565    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tm4dr" podStartSLOduration=16.669148638 podStartE2EDuration="19.208545776s" totalImagesPullingTime="2.539397138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:08:59.208221161 +0000 UTC m=+53.786918605" watchObservedRunningTime="2026-07-29 00:08:59.208545776 +0000 UTC m=+53.787243225"
Jul 29 00:09:06.098968 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:06.098942    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77kdj"
Jul 29 00:09:09.199580 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.199550    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tm4dr"
Jul 29 00:09:09.612588 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.612552    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"]
Jul 29 00:09:09.627212 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.627114    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"]
Jul 29 00:09:09.630437 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.630417    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.632549 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.632518    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 00:09:09.632669 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.632545    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 29 00:09:09.632917 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.632896    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 29 00:09:09.633023 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.632961    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 29 00:09:09.633023 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.632995    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"]
Jul 29 00:09:09.633153 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.632997    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 00:09:09.633308 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.633278    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 00:09:09.633485 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.633373    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 29 00:09:09.635019 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.635001    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"
Jul 29 00:09:09.636917 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.636898    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2n4gz\""
Jul 29 00:09:09.637055 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.637035    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 29 00:09:09.643721 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.643703    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"]
Jul 29 00:09:09.652184 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.652161    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"]
Jul 29 00:09:09.735051 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.735019    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/665d0825-aab7-41a2-856e-f8a9832216b3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.735225 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.735078    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-hub\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.735225 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.735161    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j84r\" (UniqueName: \"kubernetes.io/projected/60826e3d-5954-4353-b2e1-fa0a90bf4ca7-kube-api-access-2j84r\") pod \"managed-serviceaccount-addon-agent-8dffcd984-k6chn\" (UID: \"60826e3d-5954-4353-b2e1-fa0a90bf4ca7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"
Jul 29 00:09:09.735225 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.735219    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-ca\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.735393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.735270    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7gh5\" (UniqueName: \"kubernetes.io/projected/665d0825-aab7-41a2-856e-f8a9832216b3-kube-api-access-z7gh5\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.735393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.735297    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.735393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.735341    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60826e3d-5954-4353-b2e1-fa0a90bf4ca7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8dffcd984-k6chn\" (UID: \"60826e3d-5954-4353-b2e1-fa0a90bf4ca7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"
Jul 29 00:09:09.735509 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.735397    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.757479 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.757453    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-684c6f67d8-sbz2x"]
Jul 29 00:09:09.759281 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.759264    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.792128 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.792107    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-684c6f67d8-sbz2x"]
Jul 29 00:09:09.836189 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836161    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5da1ca5a-517b-4f53-b79f-261c58b25a22-ca-trust-extracted\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.836332 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836217    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7gh5\" (UniqueName: \"kubernetes.io/projected/665d0825-aab7-41a2-856e-f8a9832216b3-kube-api-access-z7gh5\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.836332 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836243    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5da1ca5a-517b-4f53-b79f-261c58b25a22-image-registry-private-configuration\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.836332 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836269    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.836332 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836304    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60826e3d-5954-4353-b2e1-fa0a90bf4ca7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8dffcd984-k6chn\" (UID: \"60826e3d-5954-4353-b2e1-fa0a90bf4ca7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"
Jul 29 00:09:09.836528 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836401    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5da1ca5a-517b-4f53-b79f-261c58b25a22-trusted-ca\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.836528 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836451    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5da1ca5a-517b-4f53-b79f-261c58b25a22-installation-pull-secrets\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.836528 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836475    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.836528 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836493    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/665d0825-aab7-41a2-856e-f8a9832216b3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.836528 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836510    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5da1ca5a-517b-4f53-b79f-261c58b25a22-registry-tls\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.836829 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836555    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-hub\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.836829 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836580    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5da1ca5a-517b-4f53-b79f-261c58b25a22-registry-certificates\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.836829 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836605    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j84r\" (UniqueName: \"kubernetes.io/projected/60826e3d-5954-4353-b2e1-fa0a90bf4ca7-kube-api-access-2j84r\") pod \"managed-serviceaccount-addon-agent-8dffcd984-k6chn\" (UID: \"60826e3d-5954-4353-b2e1-fa0a90bf4ca7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"
Jul 29 00:09:09.836829 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836690    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpppc\" (UniqueName: \"kubernetes.io/projected/5da1ca5a-517b-4f53-b79f-261c58b25a22-kube-api-access-xpppc\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.836829 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836778    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-ca\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.836829 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.836807    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5da1ca5a-517b-4f53-b79f-261c58b25a22-bound-sa-token\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.837307 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.837280    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/665d0825-aab7-41a2-856e-f8a9832216b3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.838867 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.838846    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.839080 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.839053    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/60826e3d-5954-4353-b2e1-fa0a90bf4ca7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8dffcd984-k6chn\" (UID: \"60826e3d-5954-4353-b2e1-fa0a90bf4ca7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"
Jul 29 00:09:09.839173 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.839154    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.839225 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.839205    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-hub\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.839277 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.839261    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/665d0825-aab7-41a2-856e-f8a9832216b3-ca\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.844690 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.844651    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j84r\" (UniqueName: \"kubernetes.io/projected/60826e3d-5954-4353-b2e1-fa0a90bf4ca7-kube-api-access-2j84r\") pod \"managed-serviceaccount-addon-agent-8dffcd984-k6chn\" (UID: \"60826e3d-5954-4353-b2e1-fa0a90bf4ca7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"
Jul 29 00:09:09.845099 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.845078    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7gh5\" (UniqueName: \"kubernetes.io/projected/665d0825-aab7-41a2-856e-f8a9832216b3-kube-api-access-z7gh5\") pod \"cluster-proxy-proxy-agent-79d8d7df56-khnn9\" (UID: \"665d0825-aab7-41a2-856e-f8a9832216b3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.937229 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.937155    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5da1ca5a-517b-4f53-b79f-261c58b25a22-registry-certificates\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.937229 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.937197    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpppc\" (UniqueName: \"kubernetes.io/projected/5da1ca5a-517b-4f53-b79f-261c58b25a22-kube-api-access-xpppc\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.937419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.937228    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5da1ca5a-517b-4f53-b79f-261c58b25a22-bound-sa-token\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.937419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.937255    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5da1ca5a-517b-4f53-b79f-261c58b25a22-ca-trust-extracted\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.937419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.937331    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5da1ca5a-517b-4f53-b79f-261c58b25a22-image-registry-private-configuration\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.937419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.937370    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5da1ca5a-517b-4f53-b79f-261c58b25a22-trusted-ca\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.937419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.937412    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5da1ca5a-517b-4f53-b79f-261c58b25a22-installation-pull-secrets\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.937693 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.937443    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5da1ca5a-517b-4f53-b79f-261c58b25a22-registry-tls\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.937755 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.937701    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5da1ca5a-517b-4f53-b79f-261c58b25a22-ca-trust-extracted\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.938037 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.938008    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5da1ca5a-517b-4f53-b79f-261c58b25a22-registry-certificates\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.938441 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.938417    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5da1ca5a-517b-4f53-b79f-261c58b25a22-trusted-ca\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.939795 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.939775    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5da1ca5a-517b-4f53-b79f-261c58b25a22-registry-tls\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.939914 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.939863    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5da1ca5a-517b-4f53-b79f-261c58b25a22-image-registry-private-configuration\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.940148 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.940121    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5da1ca5a-517b-4f53-b79f-261c58b25a22-installation-pull-secrets\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.947359 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.947336    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpppc\" (UniqueName: \"kubernetes.io/projected/5da1ca5a-517b-4f53-b79f-261c58b25a22-kube-api-access-xpppc\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.947646 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.947631    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5da1ca5a-517b-4f53-b79f-261c58b25a22-bound-sa-token\") pod \"image-registry-684c6f67d8-sbz2x\" (UID: \"5da1ca5a-517b-4f53-b79f-261c58b25a22\") " pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:09.949610 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.949587    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"
Jul 29 00:09:09.956929 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:09.956912    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"
Jul 29 00:09:10.035212 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.033842    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6498b7f75f-bw9f7"]
Jul 29 00:09:10.036544 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.036519    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.038301 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.038279    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 00:09:10.038938 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.038917    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 00:09:10.039030 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.038998    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 00:09:10.039467 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.039345    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 00:09:10.039467 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.039360    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 00:09:10.039612 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.039595    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jxcb6\""
Jul 29 00:09:10.039908 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.039881    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 00:09:10.040009 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.039886    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 00:09:10.044732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.044713    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 00:09:10.047890 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.047394    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6498b7f75f-bw9f7"]
Jul 29 00:09:10.077425 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.077391    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:10.081777 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.081754    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9"]
Jul 29 00:09:10.085489 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:09:10.085461    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod665d0825_aab7_41a2_856e_f8a9832216b3.slice/crio-c6500885d0b24fad9a8cf76265d47d8d7c2603f76cc819e1296bd3c394346ea4 WatchSource:0}: Error finding container c6500885d0b24fad9a8cf76265d47d8d7c2603f76cc819e1296bd3c394346ea4: Status 404 returned error can't find the container with id c6500885d0b24fad9a8cf76265d47d8d7c2603f76cc819e1296bd3c394346ea4
Jul 29 00:09:10.095906 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.095862    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn"]
Jul 29 00:09:10.098042 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:09:10.098021    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60826e3d_5954_4353_b2e1_fa0a90bf4ca7.slice/crio-29a3ab2f17984a0df2dc61e3e1f91ac3916d1470cf9a17cd87a46099bbf36d51 WatchSource:0}: Error finding container 29a3ab2f17984a0df2dc61e3e1f91ac3916d1470cf9a17cd87a46099bbf36d51: Status 404 returned error can't find the container with id 29a3ab2f17984a0df2dc61e3e1f91ac3916d1470cf9a17cd87a46099bbf36d51
Jul 29 00:09:10.138987 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.138957    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-console-config\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.139099 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.139067    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-serving-cert\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.139099 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.139093    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-oauth-config\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.139179 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.139115    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-trusted-ca-bundle\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.139223 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.139172    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-service-ca\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.139296 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.139279    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-oauth-serving-cert\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.139348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.139332    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9zgg\" (UniqueName: \"kubernetes.io/projected/811edbad-f8cd-4235-afb1-b997419cadf8-kube-api-access-g9zgg\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.200880 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.200811    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-684c6f67d8-sbz2x"]
Jul 29 00:09:10.204090 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:09:10.204065    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da1ca5a_517b_4f53_b79f_261c58b25a22.slice/crio-6670fed053801b35213d45ac7914776442b4061a3d2ced402fc17014b8d0524f WatchSource:0}: Error finding container 6670fed053801b35213d45ac7914776442b4061a3d2ced402fc17014b8d0524f: Status 404 returned error can't find the container with id 6670fed053801b35213d45ac7914776442b4061a3d2ced402fc17014b8d0524f
Jul 29 00:09:10.228331 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.228306    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x" event={"ID":"5da1ca5a-517b-4f53-b79f-261c58b25a22","Type":"ContainerStarted","Data":"6670fed053801b35213d45ac7914776442b4061a3d2ced402fc17014b8d0524f"}
Jul 29 00:09:10.229376 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.229357    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn" event={"ID":"60826e3d-5954-4353-b2e1-fa0a90bf4ca7","Type":"ContainerStarted","Data":"29a3ab2f17984a0df2dc61e3e1f91ac3916d1470cf9a17cd87a46099bbf36d51"}
Jul 29 00:09:10.230344 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.230327    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9" event={"ID":"665d0825-aab7-41a2-856e-f8a9832216b3","Type":"ContainerStarted","Data":"c6500885d0b24fad9a8cf76265d47d8d7c2603f76cc819e1296bd3c394346ea4"}
Jul 29 00:09:10.239692 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.239670    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9zgg\" (UniqueName: \"kubernetes.io/projected/811edbad-f8cd-4235-afb1-b997419cadf8-kube-api-access-g9zgg\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.239769 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.239716    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-console-config\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.239769 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.239763    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-serving-cert\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.239875 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.239779    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-oauth-config\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.239875 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.239799    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-trusted-ca-bundle\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.239875 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.239828    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-service-ca\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.240018 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.239887    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-oauth-serving-cert\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.240926 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.240633    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-oauth-serving-cert\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.240926 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.240733    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-service-ca\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.240926 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.240839    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-trusted-ca-bundle\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.241184 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.241104    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-console-config\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.243081 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.243064    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-serving-cert\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.243165 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.243122    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-oauth-config\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.246836 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.246816    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9zgg\" (UniqueName: \"kubernetes.io/projected/811edbad-f8cd-4235-afb1-b997419cadf8-kube-api-access-g9zgg\") pod \"console-6498b7f75f-bw9f7\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") " pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.347306 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.347273    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:10.467267 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:10.467178    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6498b7f75f-bw9f7"]
Jul 29 00:09:10.471916 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:09:10.471880    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811edbad_f8cd_4235_afb1_b997419cadf8.slice/crio-b1310e16b6dc373bdd1dbf91c82732236a541b53d9ccf5fbeaeeff3508acaa3e WatchSource:0}: Error finding container b1310e16b6dc373bdd1dbf91c82732236a541b53d9ccf5fbeaeeff3508acaa3e: Status 404 returned error can't find the container with id b1310e16b6dc373bdd1dbf91c82732236a541b53d9ccf5fbeaeeff3508acaa3e
Jul 29 00:09:11.249628 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.249524    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6498b7f75f-bw9f7" event={"ID":"811edbad-f8cd-4235-afb1-b997419cadf8","Type":"ContainerStarted","Data":"b1310e16b6dc373bdd1dbf91c82732236a541b53d9ccf5fbeaeeff3508acaa3e"}
Jul 29 00:09:11.253608 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.253572    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x" event={"ID":"5da1ca5a-517b-4f53-b79f-261c58b25a22","Type":"ContainerStarted","Data":"4685147924df1c42f2c164a73f23731f283e4af4e1b16a483791149adfc010c5"}
Jul 29 00:09:11.253855 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.253833    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:11.272562 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.271796    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x" podStartSLOduration=2.271776151 podStartE2EDuration="2.271776151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:09:11.270415546 +0000 UTC m=+65.849112994" watchObservedRunningTime="2026-07-29 00:09:11.271776151 +0000 UTC m=+65.850473599"
Jul 29 00:09:11.553812 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.553271    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:09:11.555984 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.555726    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 00:09:11.571211 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.571183    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/457be539-8706-406f-b6b0-4df3128fcb60-metrics-certs\") pod \"network-metrics-daemon-xvzvw\" (UID: \"457be539-8706-406f-b6b0-4df3128fcb60\") " pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:09:11.655264 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.654727    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:09:11.656946 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.656697    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 00:09:11.674618 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.674580    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a611c2ee-2d8f-4099-ad41-d2be9a0497fa-original-pull-secret\") pod \"global-pull-secret-syncer-njnlq\" (UID: \"a611c2ee-2d8f-4099-ad41-d2be9a0497fa\") " pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:09:11.756078 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.756032    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcf7h\" (UniqueName: \"kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h\") pod \"network-check-target-jbsdb\" (UID: \"63ef51dc-018c-4a46-83b5-8f056b947beb\") " pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:09:11.758311 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.758092    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 00:09:11.768807 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.768601    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 00:09:11.788892 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.788867    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcf7h\" (UniqueName: \"kubernetes.io/projected/63ef51dc-018c-4a46-83b5-8f056b947beb-kube-api-access-wcf7h\") pod \"network-check-target-jbsdb\" (UID: \"63ef51dc-018c-4a46-83b5-8f056b947beb\") " pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:09:11.847154 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.847128    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-njnlq"
Jul 29 00:09:11.856291 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.856265    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-thdv6\""
Jul 29 00:09:11.862035 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.862014    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pdg8h\""
Jul 29 00:09:11.865438 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.865416    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvzvw"
Jul 29 00:09:11.871719 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:11.871628    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:09:12.098420 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:12.098316    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-njnlq"]
Jul 29 00:09:12.144716 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:12.144118    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xvzvw"]
Jul 29 00:09:13.137484 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:09:13.137441    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda611c2ee_2d8f_4099_ad41_d2be9a0497fa.slice/crio-9e568d0c5b5b409f40905931a4b6041d9785907019e2d75d6c3259501e319a34 WatchSource:0}: Error finding container 9e568d0c5b5b409f40905931a4b6041d9785907019e2d75d6c3259501e319a34: Status 404 returned error can't find the container with id 9e568d0c5b5b409f40905931a4b6041d9785907019e2d75d6c3259501e319a34
Jul 29 00:09:13.267545 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:13.267506    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-njnlq" event={"ID":"a611c2ee-2d8f-4099-ad41-d2be9a0497fa","Type":"ContainerStarted","Data":"9e568d0c5b5b409f40905931a4b6041d9785907019e2d75d6c3259501e319a34"}
Jul 29 00:09:13.579282 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:09:13.579211    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457be539_8706_406f_b6b0_4df3128fcb60.slice/crio-1b4d213558a726425724e3b1ab0158ef8ca919ef8533ed27d203ce3f8b9cab5f WatchSource:0}: Error finding container 1b4d213558a726425724e3b1ab0158ef8ca919ef8533ed27d203ce3f8b9cab5f: Status 404 returned error can't find the container with id 1b4d213558a726425724e3b1ab0158ef8ca919ef8533ed27d203ce3f8b9cab5f
Jul 29 00:09:14.272698 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:14.272644    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xvzvw" event={"ID":"457be539-8706-406f-b6b0-4df3128fcb60","Type":"ContainerStarted","Data":"1b4d213558a726425724e3b1ab0158ef8ca919ef8533ed27d203ce3f8b9cab5f"}
Jul 29 00:09:14.682277 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:14.682231    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:09:14.685035 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:14.685012    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e94e1019-75e2-4403-bfa0-20e6fdc8bd51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd2sr\" (UID: \"e94e1019-75e2-4403-bfa0-20e6fdc8bd51\") " pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:09:14.957985 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:14.957722    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gmgkv\""
Jul 29 00:09:14.966305 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:14.966269    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vd2sr"
Jul 29 00:09:16.133412 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:16.133319    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vd2sr"]
Jul 29 00:09:16.149009 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:16.148941    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jbsdb"]
Jul 29 00:09:16.281727 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:16.281698    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jbsdb" event={"ID":"63ef51dc-018c-4a46-83b5-8f056b947beb","Type":"ContainerStarted","Data":"edf90d597290528fbe6edce5b249cf92fddd1fd509838abdd5448b703cb6b7ed"}
Jul 29 00:09:16.283282 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:16.283244    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vd2sr" event={"ID":"e94e1019-75e2-4403-bfa0-20e6fdc8bd51","Type":"ContainerStarted","Data":"3219ce97b52e960e1e0b854591ff4dba138daa4f85d58dd0029fb04e99556ca3"}
Jul 29 00:09:17.294527 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.293348    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9" event={"ID":"665d0825-aab7-41a2-856e-f8a9832216b3","Type":"ContainerStarted","Data":"e610f072c84ab47c06ccb017cb3af2677293b13936b3601ce55facf217c1e9bf"}
Jul 29 00:09:17.294527 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.293400    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9" event={"ID":"665d0825-aab7-41a2-856e-f8a9832216b3","Type":"ContainerStarted","Data":"d673406e97a3ba2ad073a88c6fa5e420abea2a01cb565a40073ec035a684887e"}
Jul 29 00:09:17.294527 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.293420    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9" event={"ID":"665d0825-aab7-41a2-856e-f8a9832216b3","Type":"ContainerStarted","Data":"028a863eeac21b92c48d1d2c414b3cb66f97e36d2ef36bebf8259213a23ba428"}
Jul 29 00:09:17.297782 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.297102    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xvzvw" event={"ID":"457be539-8706-406f-b6b0-4df3128fcb60","Type":"ContainerStarted","Data":"f7224efa48f79b7d4135723966d7a403426e35e3945da743c138296d1ab34bbf"}
Jul 29 00:09:17.297782 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.297158    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xvzvw" event={"ID":"457be539-8706-406f-b6b0-4df3128fcb60","Type":"ContainerStarted","Data":"701ec582370750bcf20acbc05299b4e21c40437f4b4cff905f2f6e3be154ddd4"}
Jul 29 00:09:17.299370 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.299270    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn" event={"ID":"60826e3d-5954-4353-b2e1-fa0a90bf4ca7","Type":"ContainerStarted","Data":"7a75321cfcb0340ec6518c805f43f25852aaf772b3062b5530396aedfd6fae39"}
Jul 29 00:09:17.302068 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.302025    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6498b7f75f-bw9f7" event={"ID":"811edbad-f8cd-4235-afb1-b997419cadf8","Type":"ContainerStarted","Data":"bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab"}
Jul 29 00:09:17.313156 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.312011    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79d8d7df56-khnn9" podStartSLOduration=2.145291496 podStartE2EDuration="8.311993886s" totalImagesPullingTime="6.16670239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:09:17.311040321 +0000 UTC m=+71.889737797" watchObservedRunningTime="2026-07-29 00:09:17.311993886 +0000 UTC m=+71.890691332"
Jul 29 00:09:17.327890 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.326878    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8dffcd984-k6chn" podStartSLOduration=2.176636188 podStartE2EDuration="8.326862984s" totalImagesPullingTime="6.150226796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:09:17.325461017 +0000 UTC m=+71.904158464" watchObservedRunningTime="2026-07-29 00:09:17.326862984 +0000 UTC m=+71.905560431"
Jul 29 00:09:17.367699 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.367064    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6498b7f75f-bw9f7" podStartSLOduration=1.557928347 podStartE2EDuration="7.367049889s" totalImagesPullingTime="5.809121542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:09:17.366299034 +0000 UTC m=+71.944996480" watchObservedRunningTime="2026-07-29 00:09:17.367049889 +0000 UTC m=+71.945747333"
Jul 29 00:09:17.408179 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:17.408119    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xvzvw" podStartSLOduration=69.704436382 podStartE2EDuration="1m12.408099814s" totalImagesPullingTime="2.703663432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:09:17.407360308 +0000 UTC m=+71.986057755" watchObservedRunningTime="2026-07-29 00:09:17.408099814 +0000 UTC m=+71.986797261"
Jul 29 00:09:19.618987 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:19.618953    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:09:20.347907 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:20.347865    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:20.348090 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:20.347951    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:20.353499 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:20.353475    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:21.322428 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.322390    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:09:21.831819 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.831783    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-phwj2"]
Jul 29 00:09:21.863984 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.863953    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:21.865861 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.865829    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 00:09:21.866166 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.866139    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 00:09:21.866361 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.866336    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 00:09:21.866537 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.866521    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 00:09:21.866759 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.866730    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4xnhn\""
Jul 29 00:09:21.866942 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.866918    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 00:09:21.867123 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.867095    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 00:09:21.946696 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.946633    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2366eb7b-7d82-4640-9a65-556275f8b6c0-root\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:21.946870 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.946793    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-wtmp\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:21.946928 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.946881    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv4c7\" (UniqueName: \"kubernetes.io/projected/2366eb7b-7d82-4640-9a65-556275f8b6c0-kube-api-access-pv4c7\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:21.947036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.946984    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-textfile\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:21.947134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.947114    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-tls\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:21.947198 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.947183    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2366eb7b-7d82-4640-9a65-556275f8b6c0-metrics-client-ca\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:21.947282 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.947259    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:21.947357 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.947301    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2366eb7b-7d82-4640-9a65-556275f8b6c0-sys\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:21.947498 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:21.947431    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.047930 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.047893    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv4c7\" (UniqueName: \"kubernetes.io/projected/2366eb7b-7d82-4640-9a65-556275f8b6c0-kube-api-access-pv4c7\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.048106 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.047943    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-textfile\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.048106 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.047976    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-tls\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.048106 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.048003    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2366eb7b-7d82-4640-9a65-556275f8b6c0-metrics-client-ca\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.048106 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.048036    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.048106 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.048062    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2366eb7b-7d82-4640-9a65-556275f8b6c0-sys\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.048106 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.048096    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.048416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.048149    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2366eb7b-7d82-4640-9a65-556275f8b6c0-root\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.048416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.048174    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-wtmp\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.048416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.048338    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-wtmp\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.048995 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.048973    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-textfile\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.049102 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:09:22.049088    2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 29 00:09:22.049160 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:09:22.049146    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-tls podName:2366eb7b-7d82-4640-9a65-556275f8b6c0 nodeName:}" failed. No retries permitted until 2026-07-29 00:09:22.549125733 +0000 UTC m=+77.127823159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-tls") pod "node-exporter-phwj2" (UID: "2366eb7b-7d82-4640-9a65-556275f8b6c0") : secret "node-exporter-tls" not found
Jul 29 00:09:22.049808 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.049784    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2366eb7b-7d82-4640-9a65-556275f8b6c0-metrics-client-ca\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.049871 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.049859    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2366eb7b-7d82-4640-9a65-556275f8b6c0-root\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.050175 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.050152    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2366eb7b-7d82-4640-9a65-556275f8b6c0-sys\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.050286 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.050248    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.052415 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.052392    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.057518 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.057492    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv4c7\" (UniqueName: \"kubernetes.io/projected/2366eb7b-7d82-4640-9a65-556275f8b6c0-kube-api-access-pv4c7\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.551671 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.551575    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-tls\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.553765 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.553745    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2366eb7b-7d82-4640-9a65-556275f8b6c0-node-exporter-tls\") pod \"node-exporter-phwj2\" (UID: \"2366eb7b-7d82-4640-9a65-556275f8b6c0\") " pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.778969 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:22.778933    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-phwj2"
Jul 29 00:09:22.916752 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:09:22.916725    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2366eb7b_7d82_4640_9a65_556275f8b6c0.slice/crio-2db8691927c3e7053cbb6c341420e18dc27edbb3d082f3be2340eca0a327a0d2 WatchSource:0}: Error finding container 2db8691927c3e7053cbb6c341420e18dc27edbb3d082f3be2340eca0a327a0d2: Status 404 returned error can't find the container with id 2db8691927c3e7053cbb6c341420e18dc27edbb3d082f3be2340eca0a327a0d2
Jul 29 00:09:23.326706 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:23.326611    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vd2sr" event={"ID":"e94e1019-75e2-4403-bfa0-20e6fdc8bd51","Type":"ContainerStarted","Data":"61f05f3a04b02f4b72fcadb684654e7613cb30c25eedd17a719173b8e80e78ab"}
Jul 29 00:09:23.328251 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:23.328224    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-njnlq" event={"ID":"a611c2ee-2d8f-4099-ad41-d2be9a0497fa","Type":"ContainerStarted","Data":"47513ca0f754655c990594bf67b619c6f731954cadfd089e7e02e4baf13cc6cd"}
Jul 29 00:09:23.329377 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:23.329354    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-phwj2" event={"ID":"2366eb7b-7d82-4640-9a65-556275f8b6c0","Type":"ContainerStarted","Data":"2db8691927c3e7053cbb6c341420e18dc27edbb3d082f3be2340eca0a327a0d2"}
Jul 29 00:09:23.330781 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:23.330759    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jbsdb" event={"ID":"63ef51dc-018c-4a46-83b5-8f056b947beb","Type":"ContainerStarted","Data":"142db06e601c0995b20a141f561a827bc51edfb0a2c7073774f290f039a2c6e1"}
Jul 29 00:09:23.330898 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:23.330881    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:09:23.341996 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:23.341951    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-njnlq" podStartSLOduration=67.570705191 podStartE2EDuration="1m17.341937715s" totalImagesPullingTime="9.771232524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:09:23.341504588 +0000 UTC m=+77.920202034" watchObservedRunningTime="2026-07-29 00:09:23.341937715 +0000 UTC m=+77.920635159"
Jul 29 00:09:23.355238 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:23.355195    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jbsdb" podStartSLOduration=70.700992345 podStartE2EDuration="1m17.355183219s" totalImagesPullingTime="6.654190874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:08:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:09:23.353802114 +0000 UTC m=+77.932499559" watchObservedRunningTime="2026-07-29 00:09:23.355183219 +0000 UTC m=+77.933880642"
Jul 29 00:09:24.337693 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:24.337643    2557 generic.go:356] "Generic (PLEG): container finished" podID="2366eb7b-7d82-4640-9a65-556275f8b6c0" containerID="d09747817ae97b1b33d66c45f839aad373bdccc93cb4a6a944004a046a93f22a" exitCode=0
Jul 29 00:09:24.338095 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:24.337731    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-phwj2" event={"ID":"2366eb7b-7d82-4640-9a65-556275f8b6c0","Type":"ContainerDied","Data":"d09747817ae97b1b33d66c45f839aad373bdccc93cb4a6a944004a046a93f22a"}
Jul 29 00:09:25.344855 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:25.344818    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-phwj2" event={"ID":"2366eb7b-7d82-4640-9a65-556275f8b6c0","Type":"ContainerStarted","Data":"b87e9a7a9f47bd53a60bd7f3525401b80be98fdbbabd660dfbc98b12bdde1b3e"}
Jul 29 00:09:25.345295 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:25.344882    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-phwj2" event={"ID":"2366eb7b-7d82-4640-9a65-556275f8b6c0","Type":"ContainerStarted","Data":"965e16ee33197034e9f3b735240bff0d77bf4c6c32811f414f108a90dc949609"}
Jul 29 00:09:25.362483 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:25.362431    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-phwj2" podStartSLOduration=3.095059374 podStartE2EDuration="4.362403704s" totalImagesPullingTime="1.195869272s" totalInitContainerRuntime="71.475058ms" isStatefulPod=true podCreationTimestamp="2026-07-29 00:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:09:25.360519824 +0000 UTC m=+79.939217269" watchObservedRunningTime="2026-07-29 00:09:25.362403704 +0000 UTC m=+79.941101205"
Jul 29 00:09:27.354930 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:27.354893    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vd2sr" event={"ID":"e94e1019-75e2-4403-bfa0-20e6fdc8bd51","Type":"ContainerStarted","Data":"57752ce7999a52005b4f44a2f9d7b0c7099f6e1b30110f64bfcd70dedb6d5932"}
Jul 29 00:09:31.376284 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:31.376249    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vd2sr" event={"ID":"e94e1019-75e2-4403-bfa0-20e6fdc8bd51","Type":"ContainerStarted","Data":"f37d32274c6f3be18826c39c252e73f50977911f9bd4b5420479b1a12ac7a84c"}
Jul 29 00:09:31.392095 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:31.392051    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vd2sr" podStartSLOduration=34.918296734 podStartE2EDuration="49.392035826s" totalImagesPullingTime="14.473739092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:08:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:09:31.391037372 +0000 UTC m=+85.969734830" watchObservedRunningTime="2026-07-29 00:09:31.392035826 +0000 UTC m=+85.970733303"
Jul 29 00:09:32.267239 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:32.267209    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-684c6f67d8-sbz2x"
Jul 29 00:09:34.630844 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.630776    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq" podUID="06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" containerName="registry" containerID="cri-o://177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7" gracePeriod=30
Jul 29 00:09:34.878856 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.878835    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:09:34.953393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.953327    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls\") pod \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") "
Jul 29 00:09:34.953393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.953368    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-bound-sa-token\") pod \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") "
Jul 29 00:09:34.953393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.953389    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-installation-pull-secrets\") pod \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") "
Jul 29 00:09:34.953949 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.953578    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-ca-trust-extracted\") pod \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") "
Jul 29 00:09:34.953949 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.953620    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-trusted-ca\") pod \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") "
Jul 29 00:09:34.953949 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.953651    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-image-registry-private-configuration\") pod \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") "
Jul 29 00:09:34.953949 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.953720    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgr9k\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-kube-api-access-tgr9k\") pod \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") "
Jul 29 00:09:34.953949 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.953746    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-certificates\") pod \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\" (UID: \"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b\") "
Jul 29 00:09:34.954215 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.954044    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:09:34.954325 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.954304    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:09:34.955850 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.955770    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:09:34.956010 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.955989    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:09:34.956086 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.956029    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:09:34.956243 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.956220    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:09:34.956425 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.956401    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-kube-api-access-tgr9k" (OuterVolumeSpecName: "kube-api-access-tgr9k") pod "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b"). InnerVolumeSpecName "kube-api-access-tgr9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:09:34.963750 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:34.963717    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" (UID: "06fb17c2-a242-45bf-9d86-3cb89f2b9a9b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:09:35.055004 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.054977    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgr9k\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-kube-api-access-tgr9k\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:09:35.055004 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.055001    2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-certificates\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:09:35.055193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.055015    2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-registry-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:09:35.055193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.055027    2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-bound-sa-token\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:09:35.055193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.055039    2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-installation-pull-secrets\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:09:35.055193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.055050    2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-ca-trust-extracted\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:09:35.055193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.055060    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-trusted-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:09:35.055193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.055073    2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b-image-registry-private-configuration\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:09:35.390237 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.390207    2557 generic.go:356] "Generic (PLEG): container finished" podID="06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" containerID="177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7" exitCode=0
Jul 29 00:09:35.390407 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.390250    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"
Jul 29 00:09:35.390407 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.390283    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq" event={"ID":"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b","Type":"ContainerDied","Data":"177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7"}
Jul 29 00:09:35.390407 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.390321    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq" event={"ID":"06fb17c2-a242-45bf-9d86-3cb89f2b9a9b","Type":"ContainerDied","Data":"649a209d80a586c944d6ef12dbfdb266e12ce11cd252902ef29cd66ba728199c"}
Jul 29 00:09:35.390407 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.390340    2557 scope.go:117] "RemoveContainer" containerID="177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7"
Jul 29 00:09:35.398835 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.398811    2557 scope.go:117] "RemoveContainer" containerID="177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7"
Jul 29 00:09:35.399085 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:09:35.399066    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7\": container with ID starting with 177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7 not found: ID does not exist" containerID="177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7"
Jul 29 00:09:35.399138 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.399094    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7"} err="failed to get container status \"177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7\": rpc error: code = NotFound desc = could not find container \"177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7\": container with ID starting with 177cfd7996cbd4ff84b707216d83a70f90d4e519de987cc043f2ae34046ccdc7 not found: ID does not exist"
Jul 29 00:09:35.409009 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.408988    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"]
Jul 29 00:09:35.412702 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.412645    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cb7fdbf5d-p6gxq"]
Jul 29 00:09:35.939640 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:35.939608    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" path="/var/lib/kubelet/pods/06fb17c2-a242-45bf-9d86-3cb89f2b9a9b/volumes"
Jul 29 00:09:46.715365 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:46.715326    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6498b7f75f-bw9f7"]
Jul 29 00:09:54.340521 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:09:54.340491    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jbsdb"
Jul 29 00:10:11.734992 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:11.734946    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6498b7f75f-bw9f7" podUID="811edbad-f8cd-4235-afb1-b997419cadf8" containerName="console" containerID="cri-o://bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab" gracePeriod=15
Jul 29 00:10:11.958459 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:11.958437    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:10:12.115241 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115213    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-trusted-ca-bundle\") pod \"811edbad-f8cd-4235-afb1-b997419cadf8\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") "
Jul 29 00:10:12.115383 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115276    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-serving-cert\") pod \"811edbad-f8cd-4235-afb1-b997419cadf8\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") "
Jul 29 00:10:12.115383 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115301    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-console-config\") pod \"811edbad-f8cd-4235-afb1-b997419cadf8\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") "
Jul 29 00:10:12.115383 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115327    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-oauth-config\") pod \"811edbad-f8cd-4235-afb1-b997419cadf8\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") "
Jul 29 00:10:12.115383 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115349    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-oauth-serving-cert\") pod \"811edbad-f8cd-4235-afb1-b997419cadf8\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") "
Jul 29 00:10:12.115595 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115398    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9zgg\" (UniqueName: \"kubernetes.io/projected/811edbad-f8cd-4235-afb1-b997419cadf8-kube-api-access-g9zgg\") pod \"811edbad-f8cd-4235-afb1-b997419cadf8\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") "
Jul 29 00:10:12.115595 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115426    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-service-ca\") pod \"811edbad-f8cd-4235-afb1-b997419cadf8\" (UID: \"811edbad-f8cd-4235-afb1-b997419cadf8\") "
Jul 29 00:10:12.115920 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115627    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "811edbad-f8cd-4235-afb1-b997419cadf8" (UID: "811edbad-f8cd-4235-afb1-b997419cadf8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:10:12.115920 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115788    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-console-config" (OuterVolumeSpecName: "console-config") pod "811edbad-f8cd-4235-afb1-b997419cadf8" (UID: "811edbad-f8cd-4235-afb1-b997419cadf8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:10:12.115920 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115813    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "811edbad-f8cd-4235-afb1-b997419cadf8" (UID: "811edbad-f8cd-4235-afb1-b997419cadf8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:10:12.115920 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.115906    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-service-ca" (OuterVolumeSpecName: "service-ca") pod "811edbad-f8cd-4235-afb1-b997419cadf8" (UID: "811edbad-f8cd-4235-afb1-b997419cadf8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:10:12.117726 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.117701    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "811edbad-f8cd-4235-afb1-b997419cadf8" (UID: "811edbad-f8cd-4235-afb1-b997419cadf8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:10:12.117726 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.117700    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/811edbad-f8cd-4235-afb1-b997419cadf8-kube-api-access-g9zgg" (OuterVolumeSpecName: "kube-api-access-g9zgg") pod "811edbad-f8cd-4235-afb1-b997419cadf8" (UID: "811edbad-f8cd-4235-afb1-b997419cadf8"). InnerVolumeSpecName "kube-api-access-g9zgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:10:12.117867 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.117797    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "811edbad-f8cd-4235-afb1-b997419cadf8" (UID: "811edbad-f8cd-4235-afb1-b997419cadf8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:10:12.216380 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.216354    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9zgg\" (UniqueName: \"kubernetes.io/projected/811edbad-f8cd-4235-afb1-b997419cadf8-kube-api-access-g9zgg\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:10:12.216380 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.216378    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:10:12.216518 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.216387    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-trusted-ca-bundle\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:10:12.216518 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.216396    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:10:12.216518 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.216406    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-console-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:10:12.216518 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.216414    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/811edbad-f8cd-4235-afb1-b997419cadf8-console-oauth-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:10:12.216518 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.216422    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/811edbad-f8cd-4235-afb1-b997419cadf8-oauth-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:10:12.515135 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.515050    2557 generic.go:356] "Generic (PLEG): container finished" podID="811edbad-f8cd-4235-afb1-b997419cadf8" containerID="bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab" exitCode=0
Jul 29 00:10:12.515135 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.515084    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6498b7f75f-bw9f7" event={"ID":"811edbad-f8cd-4235-afb1-b997419cadf8","Type":"ContainerDied","Data":"bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab"}
Jul 29 00:10:12.515310 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.515139    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6498b7f75f-bw9f7" event={"ID":"811edbad-f8cd-4235-afb1-b997419cadf8","Type":"ContainerDied","Data":"b1310e16b6dc373bdd1dbf91c82732236a541b53d9ccf5fbeaeeff3508acaa3e"}
Jul 29 00:10:12.515310 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.515160    2557 scope.go:117] "RemoveContainer" containerID="bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab"
Jul 29 00:10:12.515310 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.515097    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6498b7f75f-bw9f7"
Jul 29 00:10:12.525381 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.525214    2557 scope.go:117] "RemoveContainer" containerID="bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab"
Jul 29 00:10:12.525579 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:10:12.525559    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab\": container with ID starting with bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab not found: ID does not exist" containerID="bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab"
Jul 29 00:10:12.525690 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.525590    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab"} err="failed to get container status \"bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab\": rpc error: code = NotFound desc = could not find container \"bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab\": container with ID starting with bd49ac9a1bd5c09bc47d59c08a7fb9e665d6fe519d7e129341291169f333b9ab not found: ID does not exist"
Jul 29 00:10:12.536672 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.536632    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6498b7f75f-bw9f7"]
Jul 29 00:10:12.540304 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:12.540284    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6498b7f75f-bw9f7"]
Jul 29 00:10:13.939411 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:10:13.939373    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="811edbad-f8cd-4235-afb1-b997419cadf8" path="/var/lib/kubelet/pods/811edbad-f8cd-4235-afb1-b997419cadf8/volumes"
Jul 29 00:12:28.638691 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.638606    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"]
Jul 29 00:12:28.639136 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.638867    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" containerName="registry"
Jul 29 00:12:28.639136 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.638878    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" containerName="registry"
Jul 29 00:12:28.639136 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.638900    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811edbad-f8cd-4235-afb1-b997419cadf8" containerName="console"
Jul 29 00:12:28.639136 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.638906    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="811edbad-f8cd-4235-afb1-b997419cadf8" containerName="console"
Jul 29 00:12:28.639136 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.638947    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="811edbad-f8cd-4235-afb1-b997419cadf8" containerName="console"
Jul 29 00:12:28.639136 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.638954    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="06fb17c2-a242-45bf-9d86-3cb89f2b9a9b" containerName="registry"
Jul 29 00:12:28.641583 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.641566    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"
Jul 29 00:12:28.643484 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.643460    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 00:12:28.643484 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.643474    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-9lm8t\""
Jul 29 00:12:28.643804 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.643782    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 00:12:28.650603 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.650578    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"]
Jul 29 00:12:28.707176 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.707142    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48fp9\" (UniqueName: \"kubernetes.io/projected/4b67f99f-a7d4-4d65-937d-127e2edcf5e3-kube-api-access-48fp9\") pod \"openshift-lws-operator-fd8ccff4c-gswx7\" (UID: \"4b67f99f-a7d4-4d65-937d-127e2edcf5e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"
Jul 29 00:12:28.707329 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.707186    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b67f99f-a7d4-4d65-937d-127e2edcf5e3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gswx7\" (UID: \"4b67f99f-a7d4-4d65-937d-127e2edcf5e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"
Jul 29 00:12:28.807647 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.807613    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b67f99f-a7d4-4d65-937d-127e2edcf5e3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gswx7\" (UID: \"4b67f99f-a7d4-4d65-937d-127e2edcf5e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"
Jul 29 00:12:28.807791 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.807700    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48fp9\" (UniqueName: \"kubernetes.io/projected/4b67f99f-a7d4-4d65-937d-127e2edcf5e3-kube-api-access-48fp9\") pod \"openshift-lws-operator-fd8ccff4c-gswx7\" (UID: \"4b67f99f-a7d4-4d65-937d-127e2edcf5e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"
Jul 29 00:12:28.807993 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.807971    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b67f99f-a7d4-4d65-937d-127e2edcf5e3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gswx7\" (UID: \"4b67f99f-a7d4-4d65-937d-127e2edcf5e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"
Jul 29 00:12:28.815410 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.815386    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48fp9\" (UniqueName: \"kubernetes.io/projected/4b67f99f-a7d4-4d65-937d-127e2edcf5e3-kube-api-access-48fp9\") pod \"openshift-lws-operator-fd8ccff4c-gswx7\" (UID: \"4b67f99f-a7d4-4d65-937d-127e2edcf5e3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"
Jul 29 00:12:28.950812 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:28.950725    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"
Jul 29 00:12:29.069299 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:29.069274    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7"]
Jul 29 00:12:29.072714 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:12:29.072679    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b67f99f_a7d4_4d65_937d_127e2edcf5e3.slice/crio-bdf345577e58cffebbd8b3485534acf79d1a8884af48d10f0d9d9bb890309224 WatchSource:0}: Error finding container bdf345577e58cffebbd8b3485534acf79d1a8884af48d10f0d9d9bb890309224: Status 404 returned error can't find the container with id bdf345577e58cffebbd8b3485534acf79d1a8884af48d10f0d9d9bb890309224
Jul 29 00:12:29.977948 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:29.977914    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7" event={"ID":"4b67f99f-a7d4-4d65-937d-127e2edcf5e3","Type":"ContainerStarted","Data":"bdf345577e58cffebbd8b3485534acf79d1a8884af48d10f0d9d9bb890309224"}
Jul 29 00:12:32.990550 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:32.990511    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7" event={"ID":"4b67f99f-a7d4-4d65-937d-127e2edcf5e3","Type":"ContainerStarted","Data":"8c480408e2c4406d6e6022afc3f71b632e75d85674fe7acd8184f5ff7193ba40"}
Jul 29 00:12:33.006901 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:33.006856    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gswx7" podStartSLOduration=2.030407134 podStartE2EDuration="5.006841427s" totalImagesPullingTime="2.976434293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:12:33.005082759 +0000 UTC m=+267.583780204" watchObservedRunningTime="2026-07-29 00:12:33.006841427 +0000 UTC m=+267.585538872"
Jul 29 00:12:48.511198 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.511161    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"]
Jul 29 00:12:48.515938 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.515919    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.518023 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.518005    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gwq8d\""
Jul 29 00:12:48.518354 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.518336    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 00:12:48.518442 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.518364    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 00:12:48.518818 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.518805    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 00:12:48.519167 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.519154    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 00:12:48.536807 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.536785    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"]
Jul 29 00:12:48.650924 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.650897    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dmqw\" (UniqueName: \"kubernetes.io/projected/5351c052-35c1-48b9-80a9-487c5d30a401-kube-api-access-2dmqw\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.651073 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.650948    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5351c052-35c1-48b9-80a9-487c5d30a401-tmp\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.651073 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.650987    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5351c052-35c1-48b9-80a9-487c5d30a401-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.651073 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.651014    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5351c052-35c1-48b9-80a9-487c5d30a401-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.651073 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.651038    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5351c052-35c1-48b9-80a9-487c5d30a401-manifests\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.751580 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.751544    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5351c052-35c1-48b9-80a9-487c5d30a401-tmp\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.751780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.751583    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5351c052-35c1-48b9-80a9-487c5d30a401-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.751780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.751609    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5351c052-35c1-48b9-80a9-487c5d30a401-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.751780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.751737    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5351c052-35c1-48b9-80a9-487c5d30a401-manifests\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.751934 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.751791    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dmqw\" (UniqueName: \"kubernetes.io/projected/5351c052-35c1-48b9-80a9-487c5d30a401-kube-api-access-2dmqw\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.752146 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.752125    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5351c052-35c1-48b9-80a9-487c5d30a401-manifests\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.753918 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.753884    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5351c052-35c1-48b9-80a9-487c5d30a401-tmp\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.754032 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.754017    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5351c052-35c1-48b9-80a9-487c5d30a401-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.754085 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.754066    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5351c052-35c1-48b9-80a9-487c5d30a401-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.762369 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.762312    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dmqw\" (UniqueName: \"kubernetes.io/projected/5351c052-35c1-48b9-80a9-487c5d30a401-kube-api-access-2dmqw\") pod \"opendatahub-operator-controller-manager-6fc48f86bc-plwt5\" (UID: \"5351c052-35c1-48b9-80a9-487c5d30a401\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.826053 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.826031    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:48.963002 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:48.962966    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"]
Jul 29 00:12:48.966180 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:12:48.966151    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5351c052_35c1_48b9_80a9_487c5d30a401.slice/crio-3df72d62ad133144b8608980fd81459b6fb076ebcb06491c82c912eb11837146 WatchSource:0}: Error finding container 3df72d62ad133144b8608980fd81459b6fb076ebcb06491c82c912eb11837146: Status 404 returned error can't find the container with id 3df72d62ad133144b8608980fd81459b6fb076ebcb06491c82c912eb11837146
Jul 29 00:12:49.042556 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:49.042489    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5" event={"ID":"5351c052-35c1-48b9-80a9-487c5d30a401","Type":"ContainerStarted","Data":"3df72d62ad133144b8608980fd81459b6fb076ebcb06491c82c912eb11837146"}
Jul 29 00:12:53.058933 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:53.058898    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5" event={"ID":"5351c052-35c1-48b9-80a9-487c5d30a401","Type":"ContainerStarted","Data":"098a0fd6ea217fefcf74b4f63862c51a63de6f5a5dc90106c6a31a69cd80109d"}
Jul 29 00:12:54.063517 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:54.063484    2557 generic.go:356] "Generic (PLEG): container finished" podID="5351c052-35c1-48b9-80a9-487c5d30a401" containerID="098a0fd6ea217fefcf74b4f63862c51a63de6f5a5dc90106c6a31a69cd80109d" exitCode=0
Jul 29 00:12:54.063517 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:54.063522    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5" event={"ID":"5351c052-35c1-48b9-80a9-487c5d30a401","Type":"ContainerDied","Data":"098a0fd6ea217fefcf74b4f63862c51a63de6f5a5dc90106c6a31a69cd80109d"}
Jul 29 00:12:55.068610 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:55.068572    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5" event={"ID":"5351c052-35c1-48b9-80a9-487c5d30a401","Type":"ContainerStarted","Data":"c1c27876b112b11d859df71f9ac2395125a7ed896b002e83b01581d00d637518"}
Jul 29 00:12:55.069023 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:55.068700    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:12:55.088147 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:12:55.088100    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5" podStartSLOduration=2.495624774 podStartE2EDuration="7.088082313s" totalImagesPullingTime="3.530507153s" totalInitContainerRuntime="1.061950386s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:12:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:12:55.087363146 +0000 UTC m=+289.666060590" watchObservedRunningTime="2026-07-29 00:12:55.088082313 +0000 UTC m=+289.666779759"
Jul 29 00:13:05.762251 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:05.762220    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:13:05.763162 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:05.763137    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:13:05.768334 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:05.768316    2557 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 00:13:06.074373 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:06.074191    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6fc48f86bc-plwt5"
Jul 29 00:13:17.900915 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:17.900885    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"]
Jul 29 00:13:17.904527 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:17.904497    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"
Jul 29 00:13:17.907422 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:17.907395    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 29 00:13:17.907716 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:17.907696    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5j54v\""
Jul 29 00:13:17.914067 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:17.910566    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"]
Jul 29 00:13:17.914892 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:17.914871    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:17.918027 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:17.917636    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"]
Jul 29 00:13:17.918958 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:17.918902    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rr7ks\""
Jul 29 00:13:17.919391 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:17.919372    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 29 00:13:17.924960 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:17.924927    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"]
Jul 29 00:13:18.055236 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.055197    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wxq2\" (UniqueName: \"kubernetes.io/projected/77256c38-1517-4492-a989-35a7957b1989-kube-api-access-5wxq2\") pod \"llmisvc-controller-manager-777c6bd7d5-qx7pl\" (UID: \"77256c38-1517-4492-a989-35a7957b1989\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:18.055236 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.055243    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77256c38-1517-4492-a989-35a7957b1989-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qx7pl\" (UID: \"77256c38-1517-4492-a989-35a7957b1989\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:18.055481 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.055268    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4ddc\" (UniqueName: \"kubernetes.io/projected/bf9b582a-ae98-4d10-82d6-91bef055b4d0-kube-api-access-l4ddc\") pod \"kserve-controller-manager-d6f4b65b8-czwkv\" (UID: \"bf9b582a-ae98-4d10-82d6-91bef055b4d0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"
Jul 29 00:13:18.055481 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.055323    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf9b582a-ae98-4d10-82d6-91bef055b4d0-cert\") pod \"kserve-controller-manager-d6f4b65b8-czwkv\" (UID: \"bf9b582a-ae98-4d10-82d6-91bef055b4d0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"
Jul 29 00:13:18.155822 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.155736    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf9b582a-ae98-4d10-82d6-91bef055b4d0-cert\") pod \"kserve-controller-manager-d6f4b65b8-czwkv\" (UID: \"bf9b582a-ae98-4d10-82d6-91bef055b4d0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"
Jul 29 00:13:18.155989 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.155823    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wxq2\" (UniqueName: \"kubernetes.io/projected/77256c38-1517-4492-a989-35a7957b1989-kube-api-access-5wxq2\") pod \"llmisvc-controller-manager-777c6bd7d5-qx7pl\" (UID: \"77256c38-1517-4492-a989-35a7957b1989\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:18.155989 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.155853    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77256c38-1517-4492-a989-35a7957b1989-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qx7pl\" (UID: \"77256c38-1517-4492-a989-35a7957b1989\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:18.155989 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.155879    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4ddc\" (UniqueName: \"kubernetes.io/projected/bf9b582a-ae98-4d10-82d6-91bef055b4d0-kube-api-access-l4ddc\") pod \"kserve-controller-manager-d6f4b65b8-czwkv\" (UID: \"bf9b582a-ae98-4d10-82d6-91bef055b4d0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"
Jul 29 00:13:18.156149 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:13:18.155996    2557 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 29 00:13:18.156149 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:13:18.156071    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77256c38-1517-4492-a989-35a7957b1989-cert podName:77256c38-1517-4492-a989-35a7957b1989 nodeName:}" failed. No retries permitted until 2026-07-29 00:13:18.65604966 +0000 UTC m=+313.234747085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/77256c38-1517-4492-a989-35a7957b1989-cert") pod "llmisvc-controller-manager-777c6bd7d5-qx7pl" (UID: "77256c38-1517-4492-a989-35a7957b1989") : secret "llmisvc-webhook-server-cert" not found
Jul 29 00:13:18.158204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.158181    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf9b582a-ae98-4d10-82d6-91bef055b4d0-cert\") pod \"kserve-controller-manager-d6f4b65b8-czwkv\" (UID: \"bf9b582a-ae98-4d10-82d6-91bef055b4d0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"
Jul 29 00:13:18.172969 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.172946    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4ddc\" (UniqueName: \"kubernetes.io/projected/bf9b582a-ae98-4d10-82d6-91bef055b4d0-kube-api-access-l4ddc\") pod \"kserve-controller-manager-d6f4b65b8-czwkv\" (UID: \"bf9b582a-ae98-4d10-82d6-91bef055b4d0\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"
Jul 29 00:13:18.175795 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.175772    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wxq2\" (UniqueName: \"kubernetes.io/projected/77256c38-1517-4492-a989-35a7957b1989-kube-api-access-5wxq2\") pod \"llmisvc-controller-manager-777c6bd7d5-qx7pl\" (UID: \"77256c38-1517-4492-a989-35a7957b1989\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:18.224343 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.224317    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"
Jul 29 00:13:18.349505 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.349474    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"]
Jul 29 00:13:18.352499 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:13:18.352467    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9b582a_ae98_4d10_82d6_91bef055b4d0.slice/crio-e0140c820509118697693d86be6c42a17dfa202704a1b9fa28a50dca012d89f1 WatchSource:0}: Error finding container e0140c820509118697693d86be6c42a17dfa202704a1b9fa28a50dca012d89f1: Status 404 returned error can't find the container with id e0140c820509118697693d86be6c42a17dfa202704a1b9fa28a50dca012d89f1
Jul 29 00:13:18.353806 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.353789    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:13:18.660166 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.660137    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77256c38-1517-4492-a989-35a7957b1989-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qx7pl\" (UID: \"77256c38-1517-4492-a989-35a7957b1989\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:18.662393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.662364    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77256c38-1517-4492-a989-35a7957b1989-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qx7pl\" (UID: \"77256c38-1517-4492-a989-35a7957b1989\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:18.728026 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.727992    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"]
Jul 29 00:13:18.732055 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.732039    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:18.734759 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.734739    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 00:13:18.735000 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.734986    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 00:13:18.735168 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.734989    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-w4rkp\""
Jul 29 00:13:18.735243 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.735220    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 00:13:18.736001 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.735985    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 00:13:18.756431 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.756408    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"]
Jul 29 00:13:18.833120 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.833077    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:18.862302 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.862263    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxnrr\" (UniqueName: \"kubernetes.io/projected/0230a81a-283c-42f5-8d91-a7a66aec73d3-kube-api-access-zxnrr\") pod \"kube-auth-proxy-6998fb56fc-xczc2\" (UID: \"0230a81a-283c-42f5-8d91-a7a66aec73d3\") " pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:18.862454 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.862371    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0230a81a-283c-42f5-8d91-a7a66aec73d3-tls-certs\") pod \"kube-auth-proxy-6998fb56fc-xczc2\" (UID: \"0230a81a-283c-42f5-8d91-a7a66aec73d3\") " pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:18.862454 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.862405    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0230a81a-283c-42f5-8d91-a7a66aec73d3-tmp\") pod \"kube-auth-proxy-6998fb56fc-xczc2\" (UID: \"0230a81a-283c-42f5-8d91-a7a66aec73d3\") " pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:18.962989 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.962951    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0230a81a-283c-42f5-8d91-a7a66aec73d3-tls-certs\") pod \"kube-auth-proxy-6998fb56fc-xczc2\" (UID: \"0230a81a-283c-42f5-8d91-a7a66aec73d3\") " pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:18.963410 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.963001    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0230a81a-283c-42f5-8d91-a7a66aec73d3-tmp\") pod \"kube-auth-proxy-6998fb56fc-xczc2\" (UID: \"0230a81a-283c-42f5-8d91-a7a66aec73d3\") " pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:18.963410 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.963060    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxnrr\" (UniqueName: \"kubernetes.io/projected/0230a81a-283c-42f5-8d91-a7a66aec73d3-kube-api-access-zxnrr\") pod \"kube-auth-proxy-6998fb56fc-xczc2\" (UID: \"0230a81a-283c-42f5-8d91-a7a66aec73d3\") " pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:18.964207 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.964171    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"]
Jul 29 00:13:18.965406 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.965385    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0230a81a-283c-42f5-8d91-a7a66aec73d3-tmp\") pod \"kube-auth-proxy-6998fb56fc-xczc2\" (UID: \"0230a81a-283c-42f5-8d91-a7a66aec73d3\") " pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:18.965761 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.965742    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0230a81a-283c-42f5-8d91-a7a66aec73d3-tls-certs\") pod \"kube-auth-proxy-6998fb56fc-xczc2\" (UID: \"0230a81a-283c-42f5-8d91-a7a66aec73d3\") " pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:18.967056 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:13:18.967036    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod77256c38_1517_4492_a989_35a7957b1989.slice/crio-551f42cc9ec92daf27ab80e1ea35d5d6b4485780685b428445fd5a8364ef1a9d WatchSource:0}: Error finding container 551f42cc9ec92daf27ab80e1ea35d5d6b4485780685b428445fd5a8364ef1a9d: Status 404 returned error can't find the container with id 551f42cc9ec92daf27ab80e1ea35d5d6b4485780685b428445fd5a8364ef1a9d
Jul 29 00:13:18.975039 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:18.975013    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxnrr\" (UniqueName: \"kubernetes.io/projected/0230a81a-283c-42f5-8d91-a7a66aec73d3-kube-api-access-zxnrr\") pod \"kube-auth-proxy-6998fb56fc-xczc2\" (UID: \"0230a81a-283c-42f5-8d91-a7a66aec73d3\") " pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:19.042002 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:19.041976    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"
Jul 29 00:13:19.154626 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:19.154584    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl" event={"ID":"77256c38-1517-4492-a989-35a7957b1989","Type":"ContainerStarted","Data":"551f42cc9ec92daf27ab80e1ea35d5d6b4485780685b428445fd5a8364ef1a9d"}
Jul 29 00:13:19.156759 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:19.156697    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv" event={"ID":"bf9b582a-ae98-4d10-82d6-91bef055b4d0","Type":"ContainerStarted","Data":"e0140c820509118697693d86be6c42a17dfa202704a1b9fa28a50dca012d89f1"}
Jul 29 00:13:19.202844 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:19.202754    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2"]
Jul 29 00:13:19.205584 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:13:19.205551    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0230a81a_283c_42f5_8d91_a7a66aec73d3.slice/crio-6c28e26d1afd463d95246a1af5ed59fc2d7e5f13354eac3b0af2a50f67abb3f7 WatchSource:0}: Error finding container 6c28e26d1afd463d95246a1af5ed59fc2d7e5f13354eac3b0af2a50f67abb3f7: Status 404 returned error can't find the container with id 6c28e26d1afd463d95246a1af5ed59fc2d7e5f13354eac3b0af2a50f67abb3f7
Jul 29 00:13:20.170576 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.170498    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2" event={"ID":"0230a81a-283c-42f5-8d91-a7a66aec73d3","Type":"ContainerStarted","Data":"6c28e26d1afd463d95246a1af5ed59fc2d7e5f13354eac3b0af2a50f67abb3f7"}
Jul 29 00:13:20.501044 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.500965    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"]
Jul 29 00:13:20.505554 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.505530    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.508477 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.508316    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 00:13:20.508477 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.508329    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 00:13:20.508477 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.508364    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 00:13:20.508477 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.508329    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ldvht\""
Jul 29 00:13:20.508751 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.508604    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 00:13:20.522836 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.522816    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"]
Jul 29 00:13:20.679251 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.679215    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35301f60-9663-4cce-ac09-e002b79110ca-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.679429 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.679265    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35301f60-9663-4cce-ac09-e002b79110ca-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.679429 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.679322    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35301f60-9663-4cce-ac09-e002b79110ca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.679429 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.679398    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwn9l\" (UniqueName: \"kubernetes.io/projected/35301f60-9663-4cce-ac09-e002b79110ca-kube-api-access-lwn9l\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.679568 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.679441    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35301f60-9663-4cce-ac09-e002b79110ca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.679568 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.679479    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35301f60-9663-4cce-ac09-e002b79110ca-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.679568 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.679516    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35301f60-9663-4cce-ac09-e002b79110ca-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.780484 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.780397    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35301f60-9663-4cce-ac09-e002b79110ca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.780484 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.780452    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwn9l\" (UniqueName: \"kubernetes.io/projected/35301f60-9663-4cce-ac09-e002b79110ca-kube-api-access-lwn9l\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.780737 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.780480    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35301f60-9663-4cce-ac09-e002b79110ca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.780737 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.780524    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35301f60-9663-4cce-ac09-e002b79110ca-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.780737 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.780555    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35301f60-9663-4cce-ac09-e002b79110ca-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.780737 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.780613    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35301f60-9663-4cce-ac09-e002b79110ca-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.780737 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.780646    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35301f60-9663-4cce-ac09-e002b79110ca-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.781160 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.781133    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35301f60-9663-4cce-ac09-e002b79110ca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.783885 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.783631    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35301f60-9663-4cce-ac09-e002b79110ca-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.783885 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.783821    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35301f60-9663-4cce-ac09-e002b79110ca-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.783885 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.783847    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35301f60-9663-4cce-ac09-e002b79110ca-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.784398 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.784372    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35301f60-9663-4cce-ac09-e002b79110ca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.791259 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.791232    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwn9l\" (UniqueName: \"kubernetes.io/projected/35301f60-9663-4cce-ac09-e002b79110ca-kube-api-access-lwn9l\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.797389 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.797360    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35301f60-9663-4cce-ac09-e002b79110ca-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-q8ckc\" (UID: \"35301f60-9663-4cce-ac09-e002b79110ca\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:20.819373 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:20.819350    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:21.369598 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:21.369553    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"]
Jul 29 00:13:21.372954 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:13:21.372924    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35301f60_9663_4cce_ac09_e002b79110ca.slice/crio-a7d3e47db55570cd4e3a048cff3e7f885da96219f6bf433fc874b7a56426ef02 WatchSource:0}: Error finding container a7d3e47db55570cd4e3a048cff3e7f885da96219f6bf433fc874b7a56426ef02: Status 404 returned error can't find the container with id a7d3e47db55570cd4e3a048cff3e7f885da96219f6bf433fc874b7a56426ef02
Jul 29 00:13:22.186517 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:22.186477    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv" event={"ID":"bf9b582a-ae98-4d10-82d6-91bef055b4d0","Type":"ContainerStarted","Data":"bbc759cf03780643aec01b24cee2e9223aa364006ddeded56da8be1b1376c9ab"}
Jul 29 00:13:22.186729 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:22.186583    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"
Jul 29 00:13:22.187874 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:22.187844    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc" event={"ID":"35301f60-9663-4cce-ac09-e002b79110ca","Type":"ContainerStarted","Data":"a7d3e47db55570cd4e3a048cff3e7f885da96219f6bf433fc874b7a56426ef02"}
Jul 29 00:13:22.202704 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:22.202647    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv" podStartSLOduration=2.347041692 podStartE2EDuration="5.202633784s" totalImagesPullingTime="2.855592092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:13:22.201266211 +0000 UTC m=+316.779963657" watchObservedRunningTime="2026-07-29 00:13:22.202633784 +0000 UTC m=+316.781331228"
Jul 29 00:13:23.192490 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:23.192446    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2" event={"ID":"0230a81a-283c-42f5-8d91-a7a66aec73d3","Type":"ContainerStarted","Data":"1978821a183fbeef6ca2655a2058ebed9ed6c1e46cb29cb97c898a3ebb78fa00"}
Jul 29 00:13:23.193926 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:23.193901    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl" event={"ID":"77256c38-1517-4492-a989-35a7957b1989","Type":"ContainerStarted","Data":"2b6fb9f673f57be6e4c19c33401efcd294babc88887640157e829dadefd110d8"}
Jul 29 00:13:23.194062 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:23.193945    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:23.210006 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:23.209966    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6998fb56fc-xczc2" podStartSLOduration=1.701297382 podStartE2EDuration="5.209952763s" totalImagesPullingTime="3.508655381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:13:23.208639828 +0000 UTC m=+317.787337272" watchObservedRunningTime="2026-07-29 00:13:23.209952763 +0000 UTC m=+317.788650236"
Jul 29 00:13:23.227241 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:23.226625    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl" podStartSLOduration=2.481657347 podStartE2EDuration="6.22660648s" totalImagesPullingTime="3.744949133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:13:23.226248343 +0000 UTC m=+317.804945789" watchObservedRunningTime="2026-07-29 00:13:23.22660648 +0000 UTC m=+317.805303926"
Jul 29 00:13:53.199348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:53.199315    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-czwkv"
Jul 29 00:13:54.200290 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:54.200261    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx7pl"
Jul 29 00:13:55.569861 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:55.569816    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:13:55.570157 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:55.569895    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:13:56.323567 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:56.323529    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc" event={"ID":"35301f60-9663-4cce-ac09-e002b79110ca","Type":"ContainerStarted","Data":"5e74089e7d97fedecfa012fa2c5da56ac609489b162564d2491a9f0a4393cfd8"}
Jul 29 00:13:56.323791 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:56.323749    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:13:56.325188 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:56.325158    2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-q8ckc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 00:13:56.325299 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:56.325216    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc" podUID="35301f60-9663-4cce-ac09-e002b79110ca" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 00:13:56.341500 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:56.341443    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc" podStartSLOduration=2.14697448 podStartE2EDuration="36.341426059s" totalImagesPullingTime="34.194451579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:13:56.340061763 +0000 UTC m=+350.918759219" watchObservedRunningTime="2026-07-29 00:13:56.341426059 +0000 UTC m=+350.920123506"
Jul 29 00:13:57.328482 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:13:57.328456    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q8ckc"
Jul 29 00:14:39.631691 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.631643    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55f65b58b5-zc8qq"]
Jul 29 00:14:39.634686 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.634647    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.636881 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.636851    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 00:14:39.641821 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.641468    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 00:14:39.641821 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.641564    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 00:14:39.641821 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.641585    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 00:14:39.641821 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.641766    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jxcb6\""
Jul 29 00:14:39.642331 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.642304    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 00:14:39.642444 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.642348    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 00:14:39.642444 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.642370    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 00:14:39.643422 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.643402    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 00:14:39.646200 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.646177    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55f65b58b5-zc8qq"]
Jul 29 00:14:39.767783 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.767738    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-console-oauth-config\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.767783 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.767783    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvdqg\" (UniqueName: \"kubernetes.io/projected/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-kube-api-access-pvdqg\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.767981 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.767805    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-trusted-ca-bundle\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.767981 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.767874    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-console-serving-cert\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.767981 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.767957    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-oauth-serving-cert\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.768082 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.767981    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-console-config\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.768082 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.768005    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-service-ca\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.868944 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.868914    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-oauth-serving-cert\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.869098 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.868949    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-console-config\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.869098 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.868969    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-service-ca\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.869098 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.868994    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-console-oauth-config\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.869098 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.869009    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvdqg\" (UniqueName: \"kubernetes.io/projected/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-kube-api-access-pvdqg\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.869098 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.869033    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-trusted-ca-bundle\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.869098 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.869060    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-console-serving-cert\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.869804 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.869777    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-oauth-serving-cert\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.869804 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.869794    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-service-ca\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.869987 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.869803    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-console-config\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.870100 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.870076    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-trusted-ca-bundle\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.871387 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.871371    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-console-serving-cert\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.871519 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.871501    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-console-oauth-config\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.888633 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.888585    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvdqg\" (UniqueName: \"kubernetes.io/projected/ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b-kube-api-access-pvdqg\") pod \"console-55f65b58b5-zc8qq\" (UID: \"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b\") " pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:39.944851 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:39.944829    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:40.072232 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:40.072201    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55f65b58b5-zc8qq"]
Jul 29 00:14:40.076211 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:14:40.076180    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd9a1b7_12b4_4629_827d_8cd4ae7c7f2b.slice/crio-72783826de48b9cec891aecd75d1a5c89b8fa719492839a84aebb71a96e399dd WatchSource:0}: Error finding container 72783826de48b9cec891aecd75d1a5c89b8fa719492839a84aebb71a96e399dd: Status 404 returned error can't find the container with id 72783826de48b9cec891aecd75d1a5c89b8fa719492839a84aebb71a96e399dd
Jul 29 00:14:40.501326 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:40.501289    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f65b58b5-zc8qq" event={"ID":"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b","Type":"ContainerStarted","Data":"96710117a42fd7e6a9337da02875cc9671e289950680225aa877c1d91281c5f1"}
Jul 29 00:14:40.501513 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:40.501348    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55f65b58b5-zc8qq" event={"ID":"ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b","Type":"ContainerStarted","Data":"72783826de48b9cec891aecd75d1a5c89b8fa719492839a84aebb71a96e399dd"}
Jul 29 00:14:40.519780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:40.519728    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55f65b58b5-zc8qq" podStartSLOduration=1.519711736 podStartE2EDuration="1.519711736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:14:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:14:40.518881136 +0000 UTC m=+395.097578580" watchObservedRunningTime="2026-07-29 00:14:40.519711736 +0000 UTC m=+395.098409181"
Jul 29 00:14:49.945632 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:49.945604    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:49.946113 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:49.945651    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:49.950705 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:49.950681    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:50.543425 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:50.543399    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55f65b58b5-zc8qq"
Jul 29 00:14:51.358878 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.358843    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"]
Jul 29 00:14:51.362134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.362118    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:14:51.365065 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.365033    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-psdv4\""
Jul 29 00:14:51.365738 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.365723    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 00:14:51.365820 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.365804    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 00:14:51.378764 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.378735    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"]
Jul 29 00:14:51.459881 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.459848    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1a42221-e836-4321-9a63-2e5d6cee636b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" (UID: \"e1a42221-e836-4321-9a63-2e5d6cee636b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:14:51.459881 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.459884    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77b4z\" (UniqueName: \"kubernetes.io/projected/e1a42221-e836-4321-9a63-2e5d6cee636b-kube-api-access-77b4z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" (UID: \"e1a42221-e836-4321-9a63-2e5d6cee636b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:14:51.560827 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.560797    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1a42221-e836-4321-9a63-2e5d6cee636b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" (UID: \"e1a42221-e836-4321-9a63-2e5d6cee636b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:14:51.560827 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.560830    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77b4z\" (UniqueName: \"kubernetes.io/projected/e1a42221-e836-4321-9a63-2e5d6cee636b-kube-api-access-77b4z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" (UID: \"e1a42221-e836-4321-9a63-2e5d6cee636b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:14:51.561183 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.561160    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1a42221-e836-4321-9a63-2e5d6cee636b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" (UID: \"e1a42221-e836-4321-9a63-2e5d6cee636b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:14:51.568079 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.568056    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77b4z\" (UniqueName: \"kubernetes.io/projected/e1a42221-e836-4321-9a63-2e5d6cee636b-kube-api-access-77b4z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" (UID: \"e1a42221-e836-4321-9a63-2e5d6cee636b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:14:51.672296 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.672217    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:14:51.806301 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:51.806247    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"]
Jul 29 00:14:51.808535 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:14:51.808508    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1a42221_e836_4321_9a63_2e5d6cee636b.slice/crio-3303ce75cde0937a857013fd55280fcc91233569f19dad41fef5936e2652fef8 WatchSource:0}: Error finding container 3303ce75cde0937a857013fd55280fcc91233569f19dad41fef5936e2652fef8: Status 404 returned error can't find the container with id 3303ce75cde0937a857013fd55280fcc91233569f19dad41fef5936e2652fef8
Jul 29 00:14:52.550627 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:52.550595    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd" event={"ID":"e1a42221-e836-4321-9a63-2e5d6cee636b","Type":"ContainerStarted","Data":"3303ce75cde0937a857013fd55280fcc91233569f19dad41fef5936e2652fef8"}
Jul 29 00:14:59.583334 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:59.583290    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd" event={"ID":"e1a42221-e836-4321-9a63-2e5d6cee636b","Type":"ContainerStarted","Data":"fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00"}
Jul 29 00:14:59.583801 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:59.583408    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:14:59.601942 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:14:59.601894    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd" podStartSLOduration=1.7286172039999999 podStartE2EDuration="8.601879396s" totalImagesPullingTime="6.873262192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:14:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:14:59.60092177 +0000 UTC m=+414.179619215" watchObservedRunningTime="2026-07-29 00:14:59.601879396 +0000 UTC m=+414.180576840"
Jul 29 00:15:10.590776 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:10.590742    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:15:12.322893 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.322860    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"]
Jul 29 00:15:12.323629 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.323575    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" containerName="manager" containerID="cri-o://fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00" gracePeriod=2
Jul 29 00:15:12.324613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.324588    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"]
Jul 29 00:15:12.325092 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:15:12.325067    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:15:12.357673 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.357634    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"]
Jul 29 00:15:12.358050 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.358035    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" containerName="manager"
Jul 29 00:15:12.358097 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.358055    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" containerName="manager"
Jul 29 00:15:12.358157 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.358145    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" containerName="manager"
Jul 29 00:15:12.361181 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.361163    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:12.362918 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:15:12.362880    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:15:12.375345 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.375322    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"]
Jul 29 00:15:12.414330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.414298    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e900026-2bc8-44d3-88d8-47128774752d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2f5xx\" (UID: \"6e900026-2bc8-44d3-88d8-47128774752d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:12.414434 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.414341    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svpvm\" (UniqueName: \"kubernetes.io/projected/6e900026-2bc8-44d3-88d8-47128774752d-kube-api-access-svpvm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2f5xx\" (UID: \"6e900026-2bc8-44d3-88d8-47128774752d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:12.515255 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.515224    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e900026-2bc8-44d3-88d8-47128774752d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2f5xx\" (UID: \"6e900026-2bc8-44d3-88d8-47128774752d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:12.515404 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.515272    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svpvm\" (UniqueName: \"kubernetes.io/projected/6e900026-2bc8-44d3-88d8-47128774752d-kube-api-access-svpvm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2f5xx\" (UID: \"6e900026-2bc8-44d3-88d8-47128774752d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:12.515638 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.515613    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e900026-2bc8-44d3-88d8-47128774752d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2f5xx\" (UID: \"6e900026-2bc8-44d3-88d8-47128774752d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:12.526284 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.526261    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svpvm\" (UniqueName: \"kubernetes.io/projected/6e900026-2bc8-44d3-88d8-47128774752d-kube-api-access-svpvm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2f5xx\" (UID: \"6e900026-2bc8-44d3-88d8-47128774752d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:12.560550 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.560529    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:15:12.562035 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:15:12.562009    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:15:12.616375 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.616339    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77b4z\" (UniqueName: \"kubernetes.io/projected/e1a42221-e836-4321-9a63-2e5d6cee636b-kube-api-access-77b4z\") pod \"e1a42221-e836-4321-9a63-2e5d6cee636b\" (UID: \"e1a42221-e836-4321-9a63-2e5d6cee636b\") "
Jul 29 00:15:12.616492 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.616393    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1a42221-e836-4321-9a63-2e5d6cee636b-extensions-socket-volume\") pod \"e1a42221-e836-4321-9a63-2e5d6cee636b\" (UID: \"e1a42221-e836-4321-9a63-2e5d6cee636b\") "
Jul 29 00:15:12.616788 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.616767    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1a42221-e836-4321-9a63-2e5d6cee636b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e1a42221-e836-4321-9a63-2e5d6cee636b" (UID: "e1a42221-e836-4321-9a63-2e5d6cee636b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:15:12.618390 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.618368    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1a42221-e836-4321-9a63-2e5d6cee636b-kube-api-access-77b4z" (OuterVolumeSpecName: "kube-api-access-77b4z") pod "e1a42221-e836-4321-9a63-2e5d6cee636b" (UID: "e1a42221-e836-4321-9a63-2e5d6cee636b"). InnerVolumeSpecName "kube-api-access-77b4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:15:12.641557 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.641534    2557 generic.go:356] "Generic (PLEG): container finished" podID="e1a42221-e836-4321-9a63-2e5d6cee636b" containerID="fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00" exitCode=0
Jul 29 00:15:12.641654 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.641585    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:15:12.641654 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.641625    2557 scope.go:117] "RemoveContainer" containerID="fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00"
Jul 29 00:15:12.643580 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:15:12.643556    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:15:12.649693 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.649671    2557 scope.go:117] "RemoveContainer" containerID="fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00"
Jul 29 00:15:12.649941 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:15:12.649923    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00\": container with ID starting with fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00 not found: ID does not exist" containerID="fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00"
Jul 29 00:15:12.650012 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.649950    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00"} err="failed to get container status \"fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00\": rpc error: code = NotFound desc = could not find container \"fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00\": container with ID starting with fadce4bc07edc6a91f96e17bf66c22adc283fa61b249a813154757cb8c0dcd00 not found: ID does not exist"
Jul 29 00:15:12.650998 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:15:12.650976    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:15:12.700258 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.700228    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:12.717160 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.717138    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1a42221-e836-4321-9a63-2e5d6cee636b-extensions-socket-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:15:12.717244 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.717163    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77b4z\" (UniqueName: \"kubernetes.io/projected/e1a42221-e836-4321-9a63-2e5d6cee636b-kube-api-access-77b4z\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:15:12.835187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:12.835160    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"]
Jul 29 00:15:12.837493 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:15:12.837463    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e900026_2bc8_44d3_88d8_47128774752d.slice/crio-a93a7f80a3068a9b30c8df08f303d73a9a34aa92de07f68e8040c2b25e1fd797 WatchSource:0}: Error finding container a93a7f80a3068a9b30c8df08f303d73a9a34aa92de07f68e8040c2b25e1fd797: Status 404 returned error can't find the container with id a93a7f80a3068a9b30c8df08f303d73a9a34aa92de07f68e8040c2b25e1fd797
Jul 29 00:15:13.648122 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:13.648080    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx" event={"ID":"6e900026-2bc8-44d3-88d8-47128774752d","Type":"ContainerStarted","Data":"762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b"}
Jul 29 00:15:13.648578 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:13.648142    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:13.648578 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:13.648170    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx" event={"ID":"6e900026-2bc8-44d3-88d8-47128774752d","Type":"ContainerStarted","Data":"a93a7f80a3068a9b30c8df08f303d73a9a34aa92de07f68e8040c2b25e1fd797"}
Jul 29 00:15:13.666825 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:15:13.666782    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hgzjd\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hgzjd"
Jul 29 00:15:13.667156 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:13.667116    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx" podStartSLOduration=1.667100242 podStartE2EDuration="1.667100242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:15:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:15:13.665467188 +0000 UTC m=+428.244164634" watchObservedRunningTime="2026-07-29 00:15:13.667100242 +0000 UTC m=+428.245797687"
Jul 29 00:15:13.944860 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:13.941282    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1a42221-e836-4321-9a63-2e5d6cee636b" path="/var/lib/kubelet/pods/e1a42221-e836-4321-9a63-2e5d6cee636b/volumes"
Jul 29 00:15:24.655264 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:24.655235    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:30.253165 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.253087    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"]
Jul 29 00:15:30.253687 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.253299    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx" podUID="6e900026-2bc8-44d3-88d8-47128774752d" containerName="manager" containerID="cri-o://762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b" gracePeriod=10
Jul 29 00:15:30.492177 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.492151    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:30.555382 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.555316    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e900026-2bc8-44d3-88d8-47128774752d-extensions-socket-volume\") pod \"6e900026-2bc8-44d3-88d8-47128774752d\" (UID: \"6e900026-2bc8-44d3-88d8-47128774752d\") "
Jul 29 00:15:30.555507 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.555382    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svpvm\" (UniqueName: \"kubernetes.io/projected/6e900026-2bc8-44d3-88d8-47128774752d-kube-api-access-svpvm\") pod \"6e900026-2bc8-44d3-88d8-47128774752d\" (UID: \"6e900026-2bc8-44d3-88d8-47128774752d\") "
Jul 29 00:15:30.555748 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.555727    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e900026-2bc8-44d3-88d8-47128774752d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6e900026-2bc8-44d3-88d8-47128774752d" (UID: "6e900026-2bc8-44d3-88d8-47128774752d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:15:30.557394 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.557367    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e900026-2bc8-44d3-88d8-47128774752d-kube-api-access-svpvm" (OuterVolumeSpecName: "kube-api-access-svpvm") pod "6e900026-2bc8-44d3-88d8-47128774752d" (UID: "6e900026-2bc8-44d3-88d8-47128774752d"). InnerVolumeSpecName "kube-api-access-svpvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:15:30.656172 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.656145    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svpvm\" (UniqueName: \"kubernetes.io/projected/6e900026-2bc8-44d3-88d8-47128774752d-kube-api-access-svpvm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:15:30.656172 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.656168    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e900026-2bc8-44d3-88d8-47128774752d-extensions-socket-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:15:30.720593 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.720561    2557 generic.go:356] "Generic (PLEG): container finished" podID="6e900026-2bc8-44d3-88d8-47128774752d" containerID="762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b" exitCode=0
Jul 29 00:15:30.720775 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.720605    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"
Jul 29 00:15:30.720775 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.720648    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx" event={"ID":"6e900026-2bc8-44d3-88d8-47128774752d","Type":"ContainerDied","Data":"762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b"}
Jul 29 00:15:30.720775 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.720726    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx" event={"ID":"6e900026-2bc8-44d3-88d8-47128774752d","Type":"ContainerDied","Data":"a93a7f80a3068a9b30c8df08f303d73a9a34aa92de07f68e8040c2b25e1fd797"}
Jul 29 00:15:30.720775 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.720758    2557 scope.go:117] "RemoveContainer" containerID="762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b"
Jul 29 00:15:30.729135 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.729117    2557 scope.go:117] "RemoveContainer" containerID="762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b"
Jul 29 00:15:30.729374 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:15:30.729358    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b\": container with ID starting with 762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b not found: ID does not exist" containerID="762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b"
Jul 29 00:15:30.729416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.729383    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b"} err="failed to get container status \"762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b\": rpc error: code = NotFound desc = could not find container \"762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b\": container with ID starting with 762b8df2aa52ee919e0628f5dfb582eefd3733372d28588ca334b1ef2c1c865b not found: ID does not exist"
Jul 29 00:15:30.740374 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.740348    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"]
Jul 29 00:15:30.746877 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:30.746856    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2f5xx"]
Jul 29 00:15:31.940298 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:31.940253    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e900026-2bc8-44d3-88d8-47128774752d" path="/var/lib/kubelet/pods/6e900026-2bc8-44d3-88d8-47128774752d/volumes"
Jul 29 00:15:46.615275 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.615236    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"]
Jul 29 00:15:46.615740 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.615544    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e900026-2bc8-44d3-88d8-47128774752d" containerName="manager"
Jul 29 00:15:46.615740 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.615557    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e900026-2bc8-44d3-88d8-47128774752d" containerName="manager"
Jul 29 00:15:46.615740 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.615617    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e900026-2bc8-44d3-88d8-47128774752d" containerName="manager"
Jul 29 00:15:46.619561 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.619541    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.627524 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.627501    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ht849\""
Jul 29 00:15:46.636645 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.636620    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"]
Jul 29 00:15:46.676781 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.676747    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3070aafb-0177-4490-8ba4-d051480325fd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.676907 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.676786    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.676907 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.676804    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.676907 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.676818    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3070aafb-0177-4490-8ba4-d051480325fd-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.676907 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.676889    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.677106 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.676946    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.677106 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.676970    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.677106 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.677033    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3070aafb-0177-4490-8ba4-d051480325fd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.677106 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.677049    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94w7n\" (UniqueName: \"kubernetes.io/projected/3070aafb-0177-4490-8ba4-d051480325fd-kube-api-access-94w7n\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778155 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778120    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3070aafb-0177-4490-8ba4-d051480325fd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778155 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778160    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778379 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778180    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778379 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778203    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3070aafb-0177-4490-8ba4-d051480325fd-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778379 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778232    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778379 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778271    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778379 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778297    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778379 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778366    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3070aafb-0177-4490-8ba4-d051480325fd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778648 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778390    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94w7n\" (UniqueName: \"kubernetes.io/projected/3070aafb-0177-4490-8ba4-d051480325fd-kube-api-access-94w7n\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778648 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778581    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778812 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778789    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778856 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778825    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.778890 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.778849    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.779177 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.779153    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3070aafb-0177-4490-8ba4-d051480325fd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.780719 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.780700    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3070aafb-0177-4490-8ba4-d051480325fd-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.781083 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.781064    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3070aafb-0177-4490-8ba4-d051480325fd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.785908 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.785886    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3070aafb-0177-4490-8ba4-d051480325fd-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.786021 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.786004    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94w7n\" (UniqueName: \"kubernetes.io/projected/3070aafb-0177-4490-8ba4-d051480325fd-kube-api-access-94w7n\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hfcwr\" (UID: \"3070aafb-0177-4490-8ba4-d051480325fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:46.929207 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:46.929130    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:47.061025 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:47.061000    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"]
Jul 29 00:15:47.063553 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:15:47.063511    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3070aafb_0177_4490_8ba4_d051480325fd.slice/crio-1d2f72bf2a9f8a00572427fe3bb46e32ff5ff174a75ca1e3180f0e2a285ce351 WatchSource:0}: Error finding container 1d2f72bf2a9f8a00572427fe3bb46e32ff5ff174a75ca1e3180f0e2a285ce351: Status 404 returned error can't find the container with id 1d2f72bf2a9f8a00572427fe3bb46e32ff5ff174a75ca1e3180f0e2a285ce351
Jul 29 00:15:47.799226 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:47.799191    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr" event={"ID":"3070aafb-0177-4490-8ba4-d051480325fd","Type":"ContainerStarted","Data":"1d2f72bf2a9f8a00572427fe3bb46e32ff5ff174a75ca1e3180f0e2a285ce351"}
Jul 29 00:15:50.163553 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:50.163512    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:15:50.163955 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:50.163597    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:15:50.163955 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:50.163643    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:15:50.813956 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:50.813920    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr" event={"ID":"3070aafb-0177-4490-8ba4-d051480325fd","Type":"ContainerStarted","Data":"f6b480ebe2b4c746f2eb02a142e82442cb5a72e8eae44a1cc346d6c59a2b6381"}
Jul 29 00:15:50.832832 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:50.832782    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr" podStartSLOduration=1.735256637 podStartE2EDuration="4.832767137s" totalImagesPullingTime="3.0975105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:15:50.830744462 +0000 UTC m=+465.409441906" watchObservedRunningTime="2026-07-29 00:15:50.832767137 +0000 UTC m=+465.411464581"
Jul 29 00:15:50.929936 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:50.929910    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:51.910573 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:51.910540    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8fszm"]
Jul 29 00:15:51.913796 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:51.913780    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:15:51.915709 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:51.915683    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 00:15:51.915827 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:51.915713    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 00:15:51.915970 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:51.915952    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 00:15:51.915970 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:51.915965    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wdk9t\""
Jul 29 00:15:51.923065 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:51.923042    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8fszm"]
Jul 29 00:15:51.934178 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:51.934154    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:52.000808 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.000774    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8fszm"]
Jul 29 00:15:52.020451 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.020421    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8swkn\" (UniqueName: \"kubernetes.io/projected/17a5475d-3c4d-46b1-93df-88786534bdf0-kube-api-access-8swkn\") pod \"limitador-limitador-64c94fd64f-8fszm\" (UID: \"17a5475d-3c4d-46b1-93df-88786534bdf0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:15:52.020554 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.020500    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17a5475d-3c4d-46b1-93df-88786534bdf0-config-file\") pod \"limitador-limitador-64c94fd64f-8fszm\" (UID: \"17a5475d-3c4d-46b1-93df-88786534bdf0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:15:52.120948 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.120912    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8swkn\" (UniqueName: \"kubernetes.io/projected/17a5475d-3c4d-46b1-93df-88786534bdf0-kube-api-access-8swkn\") pod \"limitador-limitador-64c94fd64f-8fszm\" (UID: \"17a5475d-3c4d-46b1-93df-88786534bdf0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:15:52.121111 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.121003    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17a5475d-3c4d-46b1-93df-88786534bdf0-config-file\") pod \"limitador-limitador-64c94fd64f-8fszm\" (UID: \"17a5475d-3c4d-46b1-93df-88786534bdf0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:15:52.121622 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.121595    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17a5475d-3c4d-46b1-93df-88786534bdf0-config-file\") pod \"limitador-limitador-64c94fd64f-8fszm\" (UID: \"17a5475d-3c4d-46b1-93df-88786534bdf0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:15:52.128573 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.128548    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8swkn\" (UniqueName: \"kubernetes.io/projected/17a5475d-3c4d-46b1-93df-88786534bdf0-kube-api-access-8swkn\") pod \"limitador-limitador-64c94fd64f-8fszm\" (UID: \"17a5475d-3c4d-46b1-93df-88786534bdf0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:15:52.224342 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.224276    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:15:52.346126 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.346098    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8fszm"]
Jul 29 00:15:52.348485 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:15:52.348460    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17a5475d_3c4d_46b1_93df_88786534bdf0.slice/crio-44c5ed19a307dce629bd92d2e91d9ae9e4b77e0603da458c4f45a0fa8b61ddaf WatchSource:0}: Error finding container 44c5ed19a307dce629bd92d2e91d9ae9e4b77e0603da458c4f45a0fa8b61ddaf: Status 404 returned error can't find the container with id 44c5ed19a307dce629bd92d2e91d9ae9e4b77e0603da458c4f45a0fa8b61ddaf
Jul 29 00:15:52.824225 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.824191    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm" event={"ID":"17a5475d-3c4d-46b1-93df-88786534bdf0","Type":"ContainerStarted","Data":"44c5ed19a307dce629bd92d2e91d9ae9e4b77e0603da458c4f45a0fa8b61ddaf"}
Jul 29 00:15:52.824396 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.824372    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:52.825388 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:52.825370    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:15:55.839434 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:55.839389    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm" event={"ID":"17a5475d-3c4d-46b1-93df-88786534bdf0","Type":"ContainerStarted","Data":"35ee1da0eb1450ea8b3c4b2ddb941f63d0ae2cc19d981be3500e1057e5cc341d"}
Jul 29 00:15:55.839889 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:55.839537    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:15:55.854954 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:15:55.854909    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm" podStartSLOduration=2.123532252 podStartE2EDuration="4.854892907s" totalImagesPullingTime="2.731360655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:15:55.853821412 +0000 UTC m=+470.432518856" watchObservedRunningTime="2026-07-29 00:15:55.854892907 +0000 UTC m=+470.433590352"
Jul 29 00:16:06.844786 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:06.844757    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:16:09.499860 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:09.499829    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8fszm"]
Jul 29 00:16:09.500255 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:09.500009    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm" podUID="17a5475d-3c4d-46b1-93df-88786534bdf0" containerName="limitador" containerID="cri-o://35ee1da0eb1450ea8b3c4b2ddb941f63d0ae2cc19d981be3500e1057e5cc341d" gracePeriod=30
Jul 29 00:16:09.896938 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:09.896905    2557 generic.go:356] "Generic (PLEG): container finished" podID="17a5475d-3c4d-46b1-93df-88786534bdf0" containerID="35ee1da0eb1450ea8b3c4b2ddb941f63d0ae2cc19d981be3500e1057e5cc341d" exitCode=0
Jul 29 00:16:09.897085 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:09.896982    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm" event={"ID":"17a5475d-3c4d-46b1-93df-88786534bdf0","Type":"ContainerDied","Data":"35ee1da0eb1450ea8b3c4b2ddb941f63d0ae2cc19d981be3500e1057e5cc341d"}
Jul 29 00:16:10.437342 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.437317    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:16:10.480415 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.480387    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8swkn\" (UniqueName: \"kubernetes.io/projected/17a5475d-3c4d-46b1-93df-88786534bdf0-kube-api-access-8swkn\") pod \"17a5475d-3c4d-46b1-93df-88786534bdf0\" (UID: \"17a5475d-3c4d-46b1-93df-88786534bdf0\") "
Jul 29 00:16:10.480551 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.480468    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17a5475d-3c4d-46b1-93df-88786534bdf0-config-file\") pod \"17a5475d-3c4d-46b1-93df-88786534bdf0\" (UID: \"17a5475d-3c4d-46b1-93df-88786534bdf0\") "
Jul 29 00:16:10.480815 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.480794    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17a5475d-3c4d-46b1-93df-88786534bdf0-config-file" (OuterVolumeSpecName: "config-file") pod "17a5475d-3c4d-46b1-93df-88786534bdf0" (UID: "17a5475d-3c4d-46b1-93df-88786534bdf0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:16:10.482483 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.482458    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17a5475d-3c4d-46b1-93df-88786534bdf0-kube-api-access-8swkn" (OuterVolumeSpecName: "kube-api-access-8swkn") pod "17a5475d-3c4d-46b1-93df-88786534bdf0" (UID: "17a5475d-3c4d-46b1-93df-88786534bdf0"). InnerVolumeSpecName "kube-api-access-8swkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:16:10.581639 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.581569    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8swkn\" (UniqueName: \"kubernetes.io/projected/17a5475d-3c4d-46b1-93df-88786534bdf0-kube-api-access-8swkn\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:16:10.581639 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.581593    2557 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/17a5475d-3c4d-46b1-93df-88786534bdf0-config-file\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:16:10.901900 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.901870    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm" event={"ID":"17a5475d-3c4d-46b1-93df-88786534bdf0","Type":"ContainerDied","Data":"44c5ed19a307dce629bd92d2e91d9ae9e4b77e0603da458c4f45a0fa8b61ddaf"}
Jul 29 00:16:10.902073 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.901902    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8fszm"
Jul 29 00:16:10.902073 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.901933    2557 scope.go:117] "RemoveContainer" containerID="35ee1da0eb1450ea8b3c4b2ddb941f63d0ae2cc19d981be3500e1057e5cc341d"
Jul 29 00:16:10.921383 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.921355    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8fszm"]
Jul 29 00:16:10.925157 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:10.925135    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8fszm"]
Jul 29 00:16:11.939624 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:11.939592    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17a5475d-3c4d-46b1-93df-88786534bdf0" path="/var/lib/kubelet/pods/17a5475d-3c4d-46b1-93df-88786534bdf0/volumes"
Jul 29 00:16:13.523419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.523385    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"]
Jul 29 00:16:13.523868 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.523846    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17a5475d-3c4d-46b1-93df-88786534bdf0" containerName="limitador"
Jul 29 00:16:13.523929 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.523872    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="17a5475d-3c4d-46b1-93df-88786534bdf0" containerName="limitador"
Jul 29 00:16:13.523964 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.523950    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="17a5475d-3c4d-46b1-93df-88786534bdf0" containerName="limitador"
Jul 29 00:16:13.528527 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.528507    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"
Jul 29 00:16:13.530206 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.530186    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 29 00:16:13.530206 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.530200    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lqz99\""
Jul 29 00:16:13.530358 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.530229    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 00:16:13.530589 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.530574    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 00:16:13.533269 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.533250    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"]
Jul 29 00:16:13.607936 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.607907    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cxdh\" (UniqueName: \"kubernetes.io/projected/6147e48b-b472-4a5c-b740-01fbb4ce1e8e-kube-api-access-6cxdh\") pod \"postgres-69fb4594cf-5ws8n\" (UID: \"6147e48b-b472-4a5c-b740-01fbb4ce1e8e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"
Jul 29 00:16:13.608090 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.607975    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6147e48b-b472-4a5c-b740-01fbb4ce1e8e-data\") pod \"postgres-69fb4594cf-5ws8n\" (UID: \"6147e48b-b472-4a5c-b740-01fbb4ce1e8e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"
Jul 29 00:16:13.708918 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.708887    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6147e48b-b472-4a5c-b740-01fbb4ce1e8e-data\") pod \"postgres-69fb4594cf-5ws8n\" (UID: \"6147e48b-b472-4a5c-b740-01fbb4ce1e8e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"
Jul 29 00:16:13.709083 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.708928    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cxdh\" (UniqueName: \"kubernetes.io/projected/6147e48b-b472-4a5c-b740-01fbb4ce1e8e-kube-api-access-6cxdh\") pod \"postgres-69fb4594cf-5ws8n\" (UID: \"6147e48b-b472-4a5c-b740-01fbb4ce1e8e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"
Jul 29 00:16:13.709292 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.709270    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6147e48b-b472-4a5c-b740-01fbb4ce1e8e-data\") pod \"postgres-69fb4594cf-5ws8n\" (UID: \"6147e48b-b472-4a5c-b740-01fbb4ce1e8e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"
Jul 29 00:16:13.716717 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.716697    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cxdh\" (UniqueName: \"kubernetes.io/projected/6147e48b-b472-4a5c-b740-01fbb4ce1e8e-kube-api-access-6cxdh\") pod \"postgres-69fb4594cf-5ws8n\" (UID: \"6147e48b-b472-4a5c-b740-01fbb4ce1e8e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"
Jul 29 00:16:13.840012 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.839935    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"
Jul 29 00:16:13.965440 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:13.965410    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"]
Jul 29 00:16:13.968829 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:16:13.968804    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6147e48b_b472_4a5c_b740_01fbb4ce1e8e.slice/crio-b16064beef7aced64dd874725e2f206016ef6b3d1668ae89b3d28a35892e61ca WatchSource:0}: Error finding container b16064beef7aced64dd874725e2f206016ef6b3d1668ae89b3d28a35892e61ca: Status 404 returned error can't find the container with id b16064beef7aced64dd874725e2f206016ef6b3d1668ae89b3d28a35892e61ca
Jul 29 00:16:14.927723 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:14.927685    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n" event={"ID":"6147e48b-b472-4a5c-b740-01fbb4ce1e8e","Type":"ContainerStarted","Data":"b16064beef7aced64dd874725e2f206016ef6b3d1668ae89b3d28a35892e61ca"}
Jul 29 00:16:18.946909 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:18.946820    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n" event={"ID":"6147e48b-b472-4a5c-b740-01fbb4ce1e8e","Type":"ContainerStarted","Data":"0feaa9c9c06f8967c50290e4daedc3f6c931973ad0ac5983880f316bb06ea668"}
Jul 29 00:16:18.947248 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:18.946918    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"
Jul 29 00:16:18.961020 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:18.960894    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n" podStartSLOduration=1.309623306 podStartE2EDuration="5.960873728s" totalImagesPullingTime="4.651250422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:16:18.960133916 +0000 UTC m=+493.538831364" watchObservedRunningTime="2026-07-29 00:16:18.960873728 +0000 UTC m=+493.539571175"
Jul 29 00:16:24.979268 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:24.979237    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5ws8n"
Jul 29 00:16:32.188339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.188307    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f9d5488b8-tmvf4"]
Jul 29 00:16:32.190694 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.190652    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:32.193418 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.193396    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 00:16:32.193554 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.193455    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 00:16:32.193554 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.193481    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vdjvz\""
Jul 29 00:16:32.204635 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.204612    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f9d5488b8-tmvf4"]
Jul 29 00:16:32.269431 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.269397    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a18cb117-af4c-4300-a30f-c5bd53ebc89e-webhook-cert\") pod \"maas-controller-5f9d5488b8-tmvf4\" (UID: \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\") " pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:32.269575 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.269475    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n48np\" (UniqueName: \"kubernetes.io/projected/a18cb117-af4c-4300-a30f-c5bd53ebc89e-kube-api-access-n48np\") pod \"maas-controller-5f9d5488b8-tmvf4\" (UID: \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\") " pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:32.370442 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.370398    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a18cb117-af4c-4300-a30f-c5bd53ebc89e-webhook-cert\") pod \"maas-controller-5f9d5488b8-tmvf4\" (UID: \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\") " pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:32.370584 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:16:32.370548    2557 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 29 00:16:32.370584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.370560    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n48np\" (UniqueName: \"kubernetes.io/projected/a18cb117-af4c-4300-a30f-c5bd53ebc89e-kube-api-access-n48np\") pod \"maas-controller-5f9d5488b8-tmvf4\" (UID: \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\") " pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:32.370747 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:16:32.370621    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18cb117-af4c-4300-a30f-c5bd53ebc89e-webhook-cert podName:a18cb117-af4c-4300-a30f-c5bd53ebc89e nodeName:}" failed. No retries permitted until 2026-07-29 00:16:32.870604325 +0000 UTC m=+507.449301748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/a18cb117-af4c-4300-a30f-c5bd53ebc89e-webhook-cert") pod "maas-controller-5f9d5488b8-tmvf4" (UID: "a18cb117-af4c-4300-a30f-c5bd53ebc89e") : secret "maas-controller-webhook-cert" not found
Jul 29 00:16:32.385369 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.385346    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f9bf644fc-twrgq"]
Jul 29 00:16:32.387630 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.387609    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:16:32.387805 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.387786    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n48np\" (UniqueName: \"kubernetes.io/projected/a18cb117-af4c-4300-a30f-c5bd53ebc89e-kube-api-access-n48np\") pod \"maas-controller-5f9d5488b8-tmvf4\" (UID: \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\") " pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:32.406128 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.406094    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-twrgq"]
Jul 29 00:16:32.471078 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.471004    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/101e1749-5663-44ef-8ffa-dd798f927328-webhook-cert\") pod \"maas-controller-f9bf644fc-twrgq\" (UID: \"101e1749-5663-44ef-8ffa-dd798f927328\") " pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:16:32.471078 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.471036    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x259t\" (UniqueName: \"kubernetes.io/projected/101e1749-5663-44ef-8ffa-dd798f927328-kube-api-access-x259t\") pod \"maas-controller-f9bf644fc-twrgq\" (UID: \"101e1749-5663-44ef-8ffa-dd798f927328\") " pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:16:32.572064 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.572031    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/101e1749-5663-44ef-8ffa-dd798f927328-webhook-cert\") pod \"maas-controller-f9bf644fc-twrgq\" (UID: \"101e1749-5663-44ef-8ffa-dd798f927328\") " pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:16:32.572064 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.572065    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x259t\" (UniqueName: \"kubernetes.io/projected/101e1749-5663-44ef-8ffa-dd798f927328-kube-api-access-x259t\") pod \"maas-controller-f9bf644fc-twrgq\" (UID: \"101e1749-5663-44ef-8ffa-dd798f927328\") " pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:16:32.574480 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.574450    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/101e1749-5663-44ef-8ffa-dd798f927328-webhook-cert\") pod \"maas-controller-f9bf644fc-twrgq\" (UID: \"101e1749-5663-44ef-8ffa-dd798f927328\") " pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:16:32.579302 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.579273    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x259t\" (UniqueName: \"kubernetes.io/projected/101e1749-5663-44ef-8ffa-dd798f927328-kube-api-access-x259t\") pod \"maas-controller-f9bf644fc-twrgq\" (UID: \"101e1749-5663-44ef-8ffa-dd798f927328\") " pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:16:32.702546 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.702501    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:16:32.824874 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.824809    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-twrgq"]
Jul 29 00:16:32.827302 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:16:32.827277    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod101e1749_5663_44ef_8ffa_dd798f927328.slice/crio-d5e565f875838a3ded4cc76ebfecc65da2f0a9a325ad94f24e49785ff47b2a85 WatchSource:0}: Error finding container d5e565f875838a3ded4cc76ebfecc65da2f0a9a325ad94f24e49785ff47b2a85: Status 404 returned error can't find the container with id d5e565f875838a3ded4cc76ebfecc65da2f0a9a325ad94f24e49785ff47b2a85
Jul 29 00:16:32.874695 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.874671    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a18cb117-af4c-4300-a30f-c5bd53ebc89e-webhook-cert\") pod \"maas-controller-5f9d5488b8-tmvf4\" (UID: \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\") " pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:32.876929 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:32.876908    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a18cb117-af4c-4300-a30f-c5bd53ebc89e-webhook-cert\") pod \"maas-controller-5f9d5488b8-tmvf4\" (UID: \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\") " pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:33.008902 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:33.008814    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9bf644fc-twrgq" event={"ID":"101e1749-5663-44ef-8ffa-dd798f927328","Type":"ContainerStarted","Data":"d5e565f875838a3ded4cc76ebfecc65da2f0a9a325ad94f24e49785ff47b2a85"}
Jul 29 00:16:33.100488 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:33.100463    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:33.224398 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:33.224368    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f9d5488b8-tmvf4"]
Jul 29 00:16:33.227428 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:16:33.227401    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18cb117_af4c_4300_a30f_c5bd53ebc89e.slice/crio-d149bcdd33b8a65f72c1262003b949f87cbd8f67a371ac8fd32ac304c38a1f2f WatchSource:0}: Error finding container d149bcdd33b8a65f72c1262003b949f87cbd8f67a371ac8fd32ac304c38a1f2f: Status 404 returned error can't find the container with id d149bcdd33b8a65f72c1262003b949f87cbd8f67a371ac8fd32ac304c38a1f2f
Jul 29 00:16:34.018000 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:34.017914    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4" event={"ID":"a18cb117-af4c-4300-a30f-c5bd53ebc89e","Type":"ContainerStarted","Data":"d149bcdd33b8a65f72c1262003b949f87cbd8f67a371ac8fd32ac304c38a1f2f"}
Jul 29 00:16:36.028850 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:36.028810    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9bf644fc-twrgq" event={"ID":"101e1749-5663-44ef-8ffa-dd798f927328","Type":"ContainerStarted","Data":"f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a"}
Jul 29 00:16:36.029329 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:36.028950    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:16:36.030373 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:36.030347    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4" event={"ID":"a18cb117-af4c-4300-a30f-c5bd53ebc89e","Type":"ContainerStarted","Data":"d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762"}
Jul 29 00:16:36.030512 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:36.030471    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:36.054675 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:36.049955    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f9bf644fc-twrgq" podStartSLOduration=1.397139689 podStartE2EDuration="4.049935476s" totalImagesPullingTime="2.652795787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:16:36.047582219 +0000 UTC m=+510.626279665" watchObservedRunningTime="2026-07-29 00:16:36.049935476 +0000 UTC m=+510.628632923"
Jul 29 00:16:36.072209 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:36.072163    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4" podStartSLOduration=1.810778043 podStartE2EDuration="4.072143819s" totalImagesPullingTime="2.261365776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:16:36.071518094 +0000 UTC m=+510.650215540" watchObservedRunningTime="2026-07-29 00:16:36.072143819 +0000 UTC m=+510.650841265"
Jul 29 00:16:47.036082 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.036047    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:16:47.036592 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.036127    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:47.093541 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.093501    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f9d5488b8-tmvf4"]
Jul 29 00:16:47.093727 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.093700    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4" podUID="a18cb117-af4c-4300-a30f-c5bd53ebc89e" containerName="manager" containerID="cri-o://d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762" gracePeriod=30
Jul 29 00:16:47.331057 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.331027    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:47.505384 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.505351    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a18cb117-af4c-4300-a30f-c5bd53ebc89e-webhook-cert\") pod \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\" (UID: \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\") "
Jul 29 00:16:47.505578 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.505399    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n48np\" (UniqueName: \"kubernetes.io/projected/a18cb117-af4c-4300-a30f-c5bd53ebc89e-kube-api-access-n48np\") pod \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\" (UID: \"a18cb117-af4c-4300-a30f-c5bd53ebc89e\") "
Jul 29 00:16:47.507550 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.507518    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18cb117-af4c-4300-a30f-c5bd53ebc89e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a18cb117-af4c-4300-a30f-c5bd53ebc89e" (UID: "a18cb117-af4c-4300-a30f-c5bd53ebc89e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:16:47.507644 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.507598    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18cb117-af4c-4300-a30f-c5bd53ebc89e-kube-api-access-n48np" (OuterVolumeSpecName: "kube-api-access-n48np") pod "a18cb117-af4c-4300-a30f-c5bd53ebc89e" (UID: "a18cb117-af4c-4300-a30f-c5bd53ebc89e"). InnerVolumeSpecName "kube-api-access-n48np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:16:47.606515 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.606485    2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a18cb117-af4c-4300-a30f-c5bd53ebc89e-webhook-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:16:47.606515 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:47.606514    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n48np\" (UniqueName: \"kubernetes.io/projected/a18cb117-af4c-4300-a30f-c5bd53ebc89e-kube-api-access-n48np\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:16:48.079806 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:48.079769    2557 generic.go:356] "Generic (PLEG): container finished" podID="a18cb117-af4c-4300-a30f-c5bd53ebc89e" containerID="d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762" exitCode=0
Jul 29 00:16:48.080273 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:48.079818    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4"
Jul 29 00:16:48.080273 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:48.079838    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4" event={"ID":"a18cb117-af4c-4300-a30f-c5bd53ebc89e","Type":"ContainerDied","Data":"d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762"}
Jul 29 00:16:48.080273 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:48.079880    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9d5488b8-tmvf4" event={"ID":"a18cb117-af4c-4300-a30f-c5bd53ebc89e","Type":"ContainerDied","Data":"d149bcdd33b8a65f72c1262003b949f87cbd8f67a371ac8fd32ac304c38a1f2f"}
Jul 29 00:16:48.080273 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:48.079902    2557 scope.go:117] "RemoveContainer" containerID="d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762"
Jul 29 00:16:48.087978 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:48.087961    2557 scope.go:117] "RemoveContainer" containerID="d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762"
Jul 29 00:16:48.088232 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:16:48.088213    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762\": container with ID starting with d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762 not found: ID does not exist" containerID="d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762"
Jul 29 00:16:48.088280 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:48.088241    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762"} err="failed to get container status \"d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762\": rpc error: code = NotFound desc = could not find container \"d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762\": container with ID starting with d89f2156bbf3200903e97caeacd94365ae20280fe05082a0f04eefd624186762 not found: ID does not exist"
Jul 29 00:16:48.094294 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:48.094274    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f9d5488b8-tmvf4"]
Jul 29 00:16:48.097414 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:48.097392    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f9d5488b8-tmvf4"]
Jul 29 00:16:49.941278 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:16:49.941242    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a18cb117-af4c-4300-a30f-c5bd53ebc89e" path="/var/lib/kubelet/pods/a18cb117-af4c-4300-a30f-c5bd53ebc89e/volumes"
Jul 29 00:17:08.692263 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.692226    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"]
Jul 29 00:17:08.692741 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.692542    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18cb117-af4c-4300-a30f-c5bd53ebc89e" containerName="manager"
Jul 29 00:17:08.692741 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.692553    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18cb117-af4c-4300-a30f-c5bd53ebc89e" containerName="manager"
Jul 29 00:17:08.692741 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.692614    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18cb117-af4c-4300-a30f-c5bd53ebc89e" containerName="manager"
Jul 29 00:17:08.700409 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.700377    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:08.702357 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.702319    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 29 00:17:08.702803 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.702784    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-r2s7s\""
Jul 29 00:17:08.709743 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.709717    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"]
Jul 29 00:17:08.785029 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.784997    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e8899a7-b737-435b-a63b-2c1a8b7c1b89-maas-api-tls\") pod \"maas-api-6cf8f75bb9-9d8rd\" (UID: \"1e8899a7-b737-435b-a63b-2c1a8b7c1b89\") " pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:08.785169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.785054    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-529jg\" (UniqueName: \"kubernetes.io/projected/1e8899a7-b737-435b-a63b-2c1a8b7c1b89-kube-api-access-529jg\") pod \"maas-api-6cf8f75bb9-9d8rd\" (UID: \"1e8899a7-b737-435b-a63b-2c1a8b7c1b89\") " pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:08.785169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.785110    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8899a7-b737-435b-a63b-2c1a8b7c1b89-openshift-service-ca\") pod \"maas-api-6cf8f75bb9-9d8rd\" (UID: \"1e8899a7-b737-435b-a63b-2c1a8b7c1b89\") " pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:08.886506 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.886478    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-529jg\" (UniqueName: \"kubernetes.io/projected/1e8899a7-b737-435b-a63b-2c1a8b7c1b89-kube-api-access-529jg\") pod \"maas-api-6cf8f75bb9-9d8rd\" (UID: \"1e8899a7-b737-435b-a63b-2c1a8b7c1b89\") " pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:08.886506 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.886511    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8899a7-b737-435b-a63b-2c1a8b7c1b89-openshift-service-ca\") pod \"maas-api-6cf8f75bb9-9d8rd\" (UID: \"1e8899a7-b737-435b-a63b-2c1a8b7c1b89\") " pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:08.886724 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.886559    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e8899a7-b737-435b-a63b-2c1a8b7c1b89-maas-api-tls\") pod \"maas-api-6cf8f75bb9-9d8rd\" (UID: \"1e8899a7-b737-435b-a63b-2c1a8b7c1b89\") " pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:08.887224 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.887190    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8899a7-b737-435b-a63b-2c1a8b7c1b89-openshift-service-ca\") pod \"maas-api-6cf8f75bb9-9d8rd\" (UID: \"1e8899a7-b737-435b-a63b-2c1a8b7c1b89\") " pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:08.888940 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.888919    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e8899a7-b737-435b-a63b-2c1a8b7c1b89-maas-api-tls\") pod \"maas-api-6cf8f75bb9-9d8rd\" (UID: \"1e8899a7-b737-435b-a63b-2c1a8b7c1b89\") " pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:08.897003 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:08.896980    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-529jg\" (UniqueName: \"kubernetes.io/projected/1e8899a7-b737-435b-a63b-2c1a8b7c1b89-kube-api-access-529jg\") pod \"maas-api-6cf8f75bb9-9d8rd\" (UID: \"1e8899a7-b737-435b-a63b-2c1a8b7c1b89\") " pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:09.017138 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.017061    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:09.151008 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.150951    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"]
Jul 29 00:17:09.153904 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:17:09.153860    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8899a7_b737_435b_a63b_2c1a8b7c1b89.slice/crio-bd5f1a69126c5b25ccbc3d29801576210e84cac4f4f1f200a565cccd237d2472 WatchSource:0}: Error finding container bd5f1a69126c5b25ccbc3d29801576210e84cac4f4f1f200a565cccd237d2472: Status 404 returned error can't find the container with id bd5f1a69126c5b25ccbc3d29801576210e84cac4f4f1f200a565cccd237d2472
Jul 29 00:17:09.171144 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.171115    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd" event={"ID":"1e8899a7-b737-435b-a63b-2c1a8b7c1b89","Type":"ContainerStarted","Data":"bd5f1a69126c5b25ccbc3d29801576210e84cac4f4f1f200a565cccd237d2472"}
Jul 29 00:17:09.240867 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.240833    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"]
Jul 29 00:17:09.245962 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.245946    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"
Jul 29 00:17:09.247701 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.247646    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 29 00:17:09.247827 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.247811    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2qx5w\""
Jul 29 00:17:09.254918 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.254894    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"]
Jul 29 00:17:09.261357 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.261338    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-m9sk4"]
Jul 29 00:17:09.264839 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.264823    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4"
Jul 29 00:17:09.282400 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.282379    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-m9sk4"]
Jul 29 00:17:09.390956 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.390914    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a6ffe48-47d6-4470-8d57-1b16766a4cd7-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-jtz6s\" (UID: \"2a6ffe48-47d6-4470-8d57-1b16766a4cd7\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"
Jul 29 00:17:09.391134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.390979    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fb37761-c539-4f63-9dc7-1e94976e2461-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-m9sk4\" (UID: \"0fb37761-c539-4f63-9dc7-1e94976e2461\") " pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4"
Jul 29 00:17:09.391134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.391007    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8ppf\" (UniqueName: \"kubernetes.io/projected/2a6ffe48-47d6-4470-8d57-1b16766a4cd7-kube-api-access-d8ppf\") pod \"payload-pre-processing-687dbb6cf6-jtz6s\" (UID: \"2a6ffe48-47d6-4470-8d57-1b16766a4cd7\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"
Jul 29 00:17:09.391134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.391061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdx4r\" (UniqueName: \"kubernetes.io/projected/0fb37761-c539-4f63-9dc7-1e94976e2461-kube-api-access-bdx4r\") pod \"payload-processing-5cc6cd444b-m9sk4\" (UID: \"0fb37761-c539-4f63-9dc7-1e94976e2461\") " pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4"
Jul 29 00:17:09.491755 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.491721    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a6ffe48-47d6-4470-8d57-1b16766a4cd7-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-jtz6s\" (UID: \"2a6ffe48-47d6-4470-8d57-1b16766a4cd7\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"
Jul 29 00:17:09.491910 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.491768    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fb37761-c539-4f63-9dc7-1e94976e2461-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-m9sk4\" (UID: \"0fb37761-c539-4f63-9dc7-1e94976e2461\") " pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4"
Jul 29 00:17:09.491910 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.491791    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8ppf\" (UniqueName: \"kubernetes.io/projected/2a6ffe48-47d6-4470-8d57-1b16766a4cd7-kube-api-access-d8ppf\") pod \"payload-pre-processing-687dbb6cf6-jtz6s\" (UID: \"2a6ffe48-47d6-4470-8d57-1b16766a4cd7\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"
Jul 29 00:17:09.491910 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.491826    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdx4r\" (UniqueName: \"kubernetes.io/projected/0fb37761-c539-4f63-9dc7-1e94976e2461-kube-api-access-bdx4r\") pod \"payload-processing-5cc6cd444b-m9sk4\" (UID: \"0fb37761-c539-4f63-9dc7-1e94976e2461\") " pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4"
Jul 29 00:17:09.492418 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.492396    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a6ffe48-47d6-4470-8d57-1b16766a4cd7-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-jtz6s\" (UID: \"2a6ffe48-47d6-4470-8d57-1b16766a4cd7\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"
Jul 29 00:17:09.492531 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.492511    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fb37761-c539-4f63-9dc7-1e94976e2461-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-m9sk4\" (UID: \"0fb37761-c539-4f63-9dc7-1e94976e2461\") " pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4"
Jul 29 00:17:09.499166 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.499144    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdx4r\" (UniqueName: \"kubernetes.io/projected/0fb37761-c539-4f63-9dc7-1e94976e2461-kube-api-access-bdx4r\") pod \"payload-processing-5cc6cd444b-m9sk4\" (UID: \"0fb37761-c539-4f63-9dc7-1e94976e2461\") " pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4"
Jul 29 00:17:09.499273 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.499180    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8ppf\" (UniqueName: \"kubernetes.io/projected/2a6ffe48-47d6-4470-8d57-1b16766a4cd7-kube-api-access-d8ppf\") pod \"payload-pre-processing-687dbb6cf6-jtz6s\" (UID: \"2a6ffe48-47d6-4470-8d57-1b16766a4cd7\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"
Jul 29 00:17:09.557179 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.557120    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"
Jul 29 00:17:09.574546 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.574523    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4"
Jul 29 00:17:09.693363 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.693273    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"]
Jul 29 00:17:09.696306 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:17:09.696275    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a6ffe48_47d6_4470_8d57_1b16766a4cd7.slice/crio-9d41109233e77124945f1cc5e948742d834fdf8974f2369e5b6235e809b3fcd0 WatchSource:0}: Error finding container 9d41109233e77124945f1cc5e948742d834fdf8974f2369e5b6235e809b3fcd0: Status 404 returned error can't find the container with id 9d41109233e77124945f1cc5e948742d834fdf8974f2369e5b6235e809b3fcd0
Jul 29 00:17:09.721126 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:09.721092    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-m9sk4"]
Jul 29 00:17:09.723503 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:17:09.723479    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb37761_c539_4f63_9dc7_1e94976e2461.slice/crio-7bb9ea074f514881d45e61cab546e112333d8ef8f1a15e61eb1650f7264dcb42 WatchSource:0}: Error finding container 7bb9ea074f514881d45e61cab546e112333d8ef8f1a15e61eb1650f7264dcb42: Status 404 returned error can't find the container with id 7bb9ea074f514881d45e61cab546e112333d8ef8f1a15e61eb1650f7264dcb42
Jul 29 00:17:10.182323 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:10.182269    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s" event={"ID":"2a6ffe48-47d6-4470-8d57-1b16766a4cd7","Type":"ContainerStarted","Data":"9d41109233e77124945f1cc5e948742d834fdf8974f2369e5b6235e809b3fcd0"}
Jul 29 00:17:10.183557 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:10.183528    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4" event={"ID":"0fb37761-c539-4f63-9dc7-1e94976e2461","Type":"ContainerStarted","Data":"7bb9ea074f514881d45e61cab546e112333d8ef8f1a15e61eb1650f7264dcb42"}
Jul 29 00:17:14.213362 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:14.213332    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd" event={"ID":"1e8899a7-b737-435b-a63b-2c1a8b7c1b89","Type":"ContainerStarted","Data":"f74eb06335c3afbcb635976f7e562ebb79a338f7e92ca16edfc389cdc8e5e300"}
Jul 29 00:17:14.213732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:14.213420    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:14.214929 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:14.214904    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s" event={"ID":"2a6ffe48-47d6-4470-8d57-1b16766a4cd7","Type":"ContainerStarted","Data":"54193ec78220494916648f7ed93e65be972bc0e2ec038e6b7a5f639b3ef9fbd3"}
Jul 29 00:17:14.215033 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:14.214969    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"
Jul 29 00:17:14.216269 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:14.216251    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4" event={"ID":"0fb37761-c539-4f63-9dc7-1e94976e2461","Type":"ContainerStarted","Data":"d202d62a401c434a8596d5a255a4dc34095bbc534475fe0105522d5691ebb686"}
Jul 29 00:17:14.216383 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:14.216371    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4"
Jul 29 00:17:14.228076 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:14.228034    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd" podStartSLOduration=2.192298134 podStartE2EDuration="6.22802196s" totalImagesPullingTime="4.035723826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:17:14.227535555 +0000 UTC m=+548.806233001" watchObservedRunningTime="2026-07-29 00:17:14.22802196 +0000 UTC m=+548.806719406"
Jul 29 00:17:14.241903 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:14.241863    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4" podStartSLOduration=1.68854537 podStartE2EDuration="5.24185027s" totalImagesPullingTime="3.5533049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:17:14.240468685 +0000 UTC m=+548.819166137" watchObservedRunningTime="2026-07-29 00:17:14.24185027 +0000 UTC m=+548.820547736"
Jul 29 00:17:14.257150 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:14.257101    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s" podStartSLOduration=1.707561952 podStartE2EDuration="5.257085172s" totalImagesPullingTime="3.54952322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:17:14.255157565 +0000 UTC m=+548.833855011" watchObservedRunningTime="2026-07-29 00:17:14.257085172 +0000 UTC m=+548.835782619"
Jul 29 00:17:17.229728 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:17.229618    2557 generic.go:356] "Generic (PLEG): container finished" podID="3070aafb-0177-4490-8ba4-d051480325fd" containerID="f6b480ebe2b4c746f2eb02a142e82442cb5a72e8eae44a1cc346d6c59a2b6381" exitCode=0
Jul 29 00:17:17.229728 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:17.229692    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr" event={"ID":"3070aafb-0177-4490-8ba4-d051480325fd","Type":"ContainerDied","Data":"f6b480ebe2b4c746f2eb02a142e82442cb5a72e8eae44a1cc346d6c59a2b6381"}
Jul 29 00:17:17.230167 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:17.230075    2557 scope.go:117] "RemoveContainer" containerID="f6b480ebe2b4c746f2eb02a142e82442cb5a72e8eae44a1cc346d6c59a2b6381"
Jul 29 00:17:17.231125 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:17.231098    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:17:17.231192 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:17.231163    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:17:17.231228 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:17.231194    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:17:18.235629 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:18.235598    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr" event={"ID":"3070aafb-0177-4490-8ba4-d051480325fd","Type":"ContainerStarted","Data":"dd5cbdc198f8697ac27a13b00a48205daeabf97fe63a9f29092b2c048791f1c6"}
Jul 29 00:17:18.929366 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:18.929330    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:17:18.930553 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:18.930524    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hfcwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Jul 29 00:17:18.930698 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:18.930574    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr" podUID="3070aafb-0177-4490-8ba4-d051480325fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Jul 29 00:17:19.930300 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:19.930265    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hfcwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Jul 29 00:17:19.930705 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:19.930319    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr" podUID="3070aafb-0177-4490-8ba4-d051480325fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Jul 29 00:17:20.225683 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:20.225586    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6cf8f75bb9-9d8rd"
Jul 29 00:17:20.930342 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:20.930307    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hfcwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Jul 29 00:17:20.930878 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:20.930364    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr" podUID="3070aafb-0177-4490-8ba4-d051480325fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Jul 29 00:17:21.930257 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:21.930222    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hfcwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Jul 29 00:17:21.930452 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:21.930284    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr" podUID="3070aafb-0177-4490-8ba4-d051480325fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Jul 29 00:17:22.825246 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:22.825212    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:17:22.934221 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:22.934188    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:17:23.257484 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:23.257456    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hfcwr"
Jul 29 00:17:25.222184 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.222137    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s" podUID="2a6ffe48-47d6-4470-8d57-1b16766a4cd7" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.31:9004: connect: connection refused"
Jul 29 00:17:25.222765 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.222747    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-m9sk4"
Jul 29 00:17:25.718018 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.717978    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"]
Jul 29 00:17:25.721217 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.720805    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.723011 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.722988    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gmvtm\""
Jul 29 00:17:25.723130 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.722988    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 29 00:17:25.723130 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.723047    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 00:17:25.723130 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.722995    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 00:17:25.729792 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.729769    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"]
Jul 29 00:17:25.839670 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.839628    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.839670 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.839672    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.839848 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.839707    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrsff\" (UniqueName: \"kubernetes.io/projected/0d88483d-5f75-49f7-9120-01f2a8c28e49-kube-api-access-lrsff\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.839848 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.839739    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.839848 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.839806    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d88483d-5f75-49f7-9120-01f2a8c28e49-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.839947 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.839862    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.940492 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.940465    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrsff\" (UniqueName: \"kubernetes.io/projected/0d88483d-5f75-49f7-9120-01f2a8c28e49-kube-api-access-lrsff\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.940687 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.940523    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.940687 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.940566    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d88483d-5f75-49f7-9120-01f2a8c28e49-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.940687 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.940618    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.940687 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.940670    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.940910 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.940691    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.941006 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.940981    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.941066 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.940998    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.941066 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.941019    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.942982 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.942958    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d88483d-5f75-49f7-9120-01f2a8c28e49-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.943166 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.943146    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d88483d-5f75-49f7-9120-01f2a8c28e49-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:25.948094 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:25.948071    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrsff\" (UniqueName: \"kubernetes.io/projected/0d88483d-5f75-49f7-9120-01f2a8c28e49-kube-api-access-lrsff\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8\" (UID: \"0d88483d-5f75-49f7-9120-01f2a8c28e49\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:26.032491 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:26.032427    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:26.162492 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:26.162458    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"]
Jul 29 00:17:26.165123 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:17:26.165093    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d88483d_5f75_49f7_9120_01f2a8c28e49.slice/crio-af8dc8828f23908ce57f6e674361c228c0407c8396aba6e1aa9d1f1cf0a3d7df WatchSource:0}: Error finding container af8dc8828f23908ce57f6e674361c228c0407c8396aba6e1aa9d1f1cf0a3d7df: Status 404 returned error can't find the container with id af8dc8828f23908ce57f6e674361c228c0407c8396aba6e1aa9d1f1cf0a3d7df
Jul 29 00:17:26.270014 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:26.269978    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8" event={"ID":"0d88483d-5f75-49f7-9120-01f2a8c28e49","Type":"ContainerStarted","Data":"af8dc8828f23908ce57f6e674361c228c0407c8396aba6e1aa9d1f1cf0a3d7df"}
Jul 29 00:17:28.279406 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:28.279370    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8" event={"ID":"0d88483d-5f75-49f7-9120-01f2a8c28e49","Type":"ContainerStarted","Data":"dfa87b20245944fb1344fbedacde0d29be35bdca90389550a2954da15010ae23"}
Jul 29 00:17:28.296958 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:28.296906    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8" podStartSLOduration=2.235693628 podStartE2EDuration="3.296887237s" totalImagesPullingTime="1.061193609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:17:28.295996696 +0000 UTC m=+562.874694141" watchObservedRunningTime="2026-07-29 00:17:28.296887237 +0000 UTC m=+562.875584683"
Jul 29 00:17:35.220760 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:35.220731    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-jtz6s"
Jul 29 00:17:36.033189 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:36.033158    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:36.033365 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:36.033214    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:36.045291 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:36.045261    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:36.324403 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:36.324315    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8"
Jul 29 00:17:42.305959 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.305921    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"]
Jul 29 00:17:42.308617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.308594    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.310542 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.310524    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 29 00:17:42.321405 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.321377    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"]
Jul 29 00:17:42.486447 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.486408    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.486618 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.486457    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-549zl\" (UniqueName: \"kubernetes.io/projected/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-kube-api-access-549zl\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.486618 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.486488    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.486618 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.486516    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.486618 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.486539    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.486795 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.486641    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.588077 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.587986    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.588224 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.588083    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.588224 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.588141    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-549zl\" (UniqueName: \"kubernetes.io/projected/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-kube-api-access-549zl\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.588224 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.588185    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.588384 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.588227    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.588384 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.588270    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.588384 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.588376    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.588585 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.588558    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.588750 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.588617    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.590241 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.590218    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.590521 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.590502    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.595586 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.595562    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-549zl\" (UniqueName: \"kubernetes.io/projected/1d5c50f4-decb-4192-8d99-9a750e5bc5ed-kube-api-access-549zl\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hhwz\" (UID: \"1d5c50f4-decb-4192-8d99-9a750e5bc5ed\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.619975 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.619952    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:42.745550 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:42.745519    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"]
Jul 29 00:17:42.747763 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:17:42.747738    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d5c50f4_decb_4192_8d99_9a750e5bc5ed.slice/crio-406b43b5086a3b5b3c203045a3b188343a29c083352e26f403babf8beb4a15a0 WatchSource:0}: Error finding container 406b43b5086a3b5b3c203045a3b188343a29c083352e26f403babf8beb4a15a0: Status 404 returned error can't find the container with id 406b43b5086a3b5b3c203045a3b188343a29c083352e26f403babf8beb4a15a0
Jul 29 00:17:43.346629 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:43.346591    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz" event={"ID":"1d5c50f4-decb-4192-8d99-9a750e5bc5ed","Type":"ContainerStarted","Data":"c36e6e57f8e60456315b70c334189994dcb8b6e94306f999e94d18073e8374e8"}
Jul 29 00:17:43.347115 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:43.346650    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz" event={"ID":"1d5c50f4-decb-4192-8d99-9a750e5bc5ed","Type":"ContainerStarted","Data":"406b43b5086a3b5b3c203045a3b188343a29c083352e26f403babf8beb4a15a0"}
Jul 29 00:17:43.365168 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:43.365082    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz" podStartSLOduration=1.185293875 podStartE2EDuration="1.365061793s" totalImagesPullingTime="179.767918ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:17:43.364503547 +0000 UTC m=+577.943200993" watchObservedRunningTime="2026-07-29 00:17:43.365061793 +0000 UTC m=+577.943759239"
Jul 29 00:17:52.620342 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:52.620301    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:52.620906 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:52.620370    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:52.632803 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:52.632782    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:17:53.400775 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:17:53.400746    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hhwz"
Jul 29 00:18:05.789465 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:05.789432    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:18:05.789999 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:05.789978    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:18:22.310521 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.310488    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"]
Jul 29 00:18:22.315199 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.315179    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.316947 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.316929    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 29 00:18:22.324932 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.324912    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"]
Jul 29 00:18:22.445090 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.445053    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.445241 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.445098    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvk46\" (UniqueName: \"kubernetes.io/projected/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-kube-api-access-bvk46\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.445241 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.445129    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.445241 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.445203    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.445241 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.445230    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.445377 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.445267    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.546119 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.546086    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvk46\" (UniqueName: \"kubernetes.io/projected/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-kube-api-access-bvk46\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.546119 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.546124    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.546303 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.546250    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.546303 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.546281    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.546378 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.546340    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.546438 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.546419    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.546519 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.546500    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.546633 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.546606    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.546738 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.546722    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.548505 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.548484    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.548781 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.548762    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.553128 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.553099    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvk46\" (UniqueName: \"kubernetes.io/projected/2b7623c1-67ae-4c29-9405-c7a9bde8ecdd-kube-api-access-bvk46\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6rf6r\" (UID: \"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.626393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.626366    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:22.755159 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.755132    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"]
Jul 29 00:18:22.758009 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:18:22.757979    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b7623c1_67ae_4c29_9405_c7a9bde8ecdd.slice/crio-19a961cdda168004a5065467c821dc13743bae275b232b4068fd2b2f3641b43b WatchSource:0}: Error finding container 19a961cdda168004a5065467c821dc13743bae275b232b4068fd2b2f3641b43b: Status 404 returned error can't find the container with id 19a961cdda168004a5065467c821dc13743bae275b232b4068fd2b2f3641b43b
Jul 29 00:18:22.759188 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:22.759170    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:18:23.522231 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:23.522197    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r" event={"ID":"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd","Type":"ContainerStarted","Data":"28fa6681dde3f4d059653bb18fe31b46058aebcce772d72c76b0cdabc74df31f"}
Jul 29 00:18:23.522626 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:23.522260    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r" event={"ID":"2b7623c1-67ae-4c29-9405-c7a9bde8ecdd","Type":"ContainerStarted","Data":"19a961cdda168004a5065467c821dc13743bae275b232b4068fd2b2f3641b43b"}
Jul 29 00:18:23.539467 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:23.539412    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r" podStartSLOduration=1.3767690080000001 podStartE2EDuration="1.539392057s" totalImagesPullingTime="162.623049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:18:23.538457462 +0000 UTC m=+618.117154907" watchObservedRunningTime="2026-07-29 00:18:23.539392057 +0000 UTC m=+618.118089504"
Jul 29 00:18:32.626721 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:32.626613    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:32.626721 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:32.626694    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:32.640182 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:32.640152    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:18:33.583402 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:18:33.583372    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6rf6r"
Jul 29 00:20:28.321524 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:28.321492    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-twrgq"]
Jul 29 00:20:28.322111 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:28.321744    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-f9bf644fc-twrgq" podUID="101e1749-5663-44ef-8ffa-dd798f927328" containerName="manager" containerID="cri-o://f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a" gracePeriod=30
Jul 29 00:20:28.567830 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:28.567808    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:20:28.642167 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:28.642137    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x259t\" (UniqueName: \"kubernetes.io/projected/101e1749-5663-44ef-8ffa-dd798f927328-kube-api-access-x259t\") pod \"101e1749-5663-44ef-8ffa-dd798f927328\" (UID: \"101e1749-5663-44ef-8ffa-dd798f927328\") "
Jul 29 00:20:28.642321 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:28.642173    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/101e1749-5663-44ef-8ffa-dd798f927328-webhook-cert\") pod \"101e1749-5663-44ef-8ffa-dd798f927328\" (UID: \"101e1749-5663-44ef-8ffa-dd798f927328\") "
Jul 29 00:20:28.644081 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:28.644055    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/101e1749-5663-44ef-8ffa-dd798f927328-kube-api-access-x259t" (OuterVolumeSpecName: "kube-api-access-x259t") pod "101e1749-5663-44ef-8ffa-dd798f927328" (UID: "101e1749-5663-44ef-8ffa-dd798f927328"). InnerVolumeSpecName "kube-api-access-x259t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:20:28.644198 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:28.644179    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/101e1749-5663-44ef-8ffa-dd798f927328-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "101e1749-5663-44ef-8ffa-dd798f927328" (UID: "101e1749-5663-44ef-8ffa-dd798f927328"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:20:28.743540 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:28.743506    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x259t\" (UniqueName: \"kubernetes.io/projected/101e1749-5663-44ef-8ffa-dd798f927328-kube-api-access-x259t\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:20:28.743540 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:28.743532    2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/101e1749-5663-44ef-8ffa-dd798f927328-webhook-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:20:29.081568 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:29.081475    2557 generic.go:356] "Generic (PLEG): container finished" podID="101e1749-5663-44ef-8ffa-dd798f927328" containerID="f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a" exitCode=0
Jul 29 00:20:29.081568 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:29.081526    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9bf644fc-twrgq"
Jul 29 00:20:29.081568 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:29.081531    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9bf644fc-twrgq" event={"ID":"101e1749-5663-44ef-8ffa-dd798f927328","Type":"ContainerDied","Data":"f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a"}
Jul 29 00:20:29.081804 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:29.081582    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9bf644fc-twrgq" event={"ID":"101e1749-5663-44ef-8ffa-dd798f927328","Type":"ContainerDied","Data":"d5e565f875838a3ded4cc76ebfecc65da2f0a9a325ad94f24e49785ff47b2a85"}
Jul 29 00:20:29.081804 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:29.081626    2557 scope.go:117] "RemoveContainer" containerID="f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a"
Jul 29 00:20:29.090695 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:29.090674    2557 scope.go:117] "RemoveContainer" containerID="f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a"
Jul 29 00:20:29.090935 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:20:29.090918    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a\": container with ID starting with f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a not found: ID does not exist" containerID="f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a"
Jul 29 00:20:29.090984 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:29.090942    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a"} err="failed to get container status \"f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a\": rpc error: code = NotFound desc = could not find container \"f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a\": container with ID starting with f0d89e344931c72d0859bcb8be8efd71f41595db70d45b67c161c445c33dc55a not found: ID does not exist"
Jul 29 00:20:29.104162 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:29.103362    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-twrgq"]
Jul 29 00:20:29.109038 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:29.109017    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-twrgq"]
Jul 29 00:20:29.940566 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:20:29.940531    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="101e1749-5663-44ef-8ffa-dd798f927328" path="/var/lib/kubelet/pods/101e1749-5663-44ef-8ffa-dd798f927328/volumes"
Jul 29 00:21:29.927781 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:29.927684    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f9bf644fc-skpl7"]
Jul 29 00:21:29.928169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:29.928071    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="101e1749-5663-44ef-8ffa-dd798f927328" containerName="manager"
Jul 29 00:21:29.928169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:29.928084    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="101e1749-5663-44ef-8ffa-dd798f927328" containerName="manager"
Jul 29 00:21:29.928169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:29.928148    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="101e1749-5663-44ef-8ffa-dd798f927328" containerName="manager"
Jul 29 00:21:29.931432 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:29.931412    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:29.933366 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:29.933348    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 00:21:29.933624 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:29.933606    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 00:21:29.933689 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:29.933681    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vdjvz\""
Jul 29 00:21:29.942266 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:29.942239    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-skpl7"]
Jul 29 00:21:30.060306 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:30.060276    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb436a2b-e048-4e84-ba17-24a49ef48cde-webhook-cert\") pod \"maas-controller-f9bf644fc-skpl7\" (UID: \"fb436a2b-e048-4e84-ba17-24a49ef48cde\") " pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:30.060479 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:30.060400    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4spsc\" (UniqueName: \"kubernetes.io/projected/fb436a2b-e048-4e84-ba17-24a49ef48cde-kube-api-access-4spsc\") pod \"maas-controller-f9bf644fc-skpl7\" (UID: \"fb436a2b-e048-4e84-ba17-24a49ef48cde\") " pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:30.161752 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:30.161713    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4spsc\" (UniqueName: \"kubernetes.io/projected/fb436a2b-e048-4e84-ba17-24a49ef48cde-kube-api-access-4spsc\") pod \"maas-controller-f9bf644fc-skpl7\" (UID: \"fb436a2b-e048-4e84-ba17-24a49ef48cde\") " pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:30.161919 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:30.161762    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb436a2b-e048-4e84-ba17-24a49ef48cde-webhook-cert\") pod \"maas-controller-f9bf644fc-skpl7\" (UID: \"fb436a2b-e048-4e84-ba17-24a49ef48cde\") " pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:30.164181 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:30.164162    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb436a2b-e048-4e84-ba17-24a49ef48cde-webhook-cert\") pod \"maas-controller-f9bf644fc-skpl7\" (UID: \"fb436a2b-e048-4e84-ba17-24a49ef48cde\") " pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:30.169029 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:30.169007    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4spsc\" (UniqueName: \"kubernetes.io/projected/fb436a2b-e048-4e84-ba17-24a49ef48cde-kube-api-access-4spsc\") pod \"maas-controller-f9bf644fc-skpl7\" (UID: \"fb436a2b-e048-4e84-ba17-24a49ef48cde\") " pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:30.243514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:30.243428    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:30.369800 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:30.369742    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-skpl7"]
Jul 29 00:21:30.372405 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:21:30.372379    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb436a2b_e048_4e84_ba17_24a49ef48cde.slice/crio-d908bf995a3b01f7e78d99ad4a744c579e43592de0adb42e030aa2ad1fbeaacc WatchSource:0}: Error finding container d908bf995a3b01f7e78d99ad4a744c579e43592de0adb42e030aa2ad1fbeaacc: Status 404 returned error can't find the container with id d908bf995a3b01f7e78d99ad4a744c579e43592de0adb42e030aa2ad1fbeaacc
Jul 29 00:21:31.352396 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:31.352358    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9bf644fc-skpl7" event={"ID":"fb436a2b-e048-4e84-ba17-24a49ef48cde","Type":"ContainerStarted","Data":"19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c"}
Jul 29 00:21:31.352822 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:31.352418    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9bf644fc-skpl7" event={"ID":"fb436a2b-e048-4e84-ba17-24a49ef48cde","Type":"ContainerStarted","Data":"d908bf995a3b01f7e78d99ad4a744c579e43592de0adb42e030aa2ad1fbeaacc"}
Jul 29 00:21:31.352822 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:31.352515    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:31.368803 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:31.368749    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f9bf644fc-skpl7" podStartSLOduration=1.897678189 podStartE2EDuration="2.368733421s" totalImagesPullingTime="471.055232ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:21:31.367388689 +0000 UTC m=+805.946086145" watchObservedRunningTime="2026-07-29 00:21:31.368733421 +0000 UTC m=+805.947430865"
Jul 29 00:21:42.359698 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:42.359639    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:47.017551 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.017514    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-skpl7"]
Jul 29 00:21:47.017976 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.017749    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-f9bf644fc-skpl7" podUID="fb436a2b-e048-4e84-ba17-24a49ef48cde" containerName="manager" containerID="cri-o://19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c" gracePeriod=30
Jul 29 00:21:47.259893 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.259870    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:47.304502 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.304434    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4spsc\" (UniqueName: \"kubernetes.io/projected/fb436a2b-e048-4e84-ba17-24a49ef48cde-kube-api-access-4spsc\") pod \"fb436a2b-e048-4e84-ba17-24a49ef48cde\" (UID: \"fb436a2b-e048-4e84-ba17-24a49ef48cde\") "
Jul 29 00:21:47.304502 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.304482    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb436a2b-e048-4e84-ba17-24a49ef48cde-webhook-cert\") pod \"fb436a2b-e048-4e84-ba17-24a49ef48cde\" (UID: \"fb436a2b-e048-4e84-ba17-24a49ef48cde\") "
Jul 29 00:21:47.306305 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.306274    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb436a2b-e048-4e84-ba17-24a49ef48cde-kube-api-access-4spsc" (OuterVolumeSpecName: "kube-api-access-4spsc") pod "fb436a2b-e048-4e84-ba17-24a49ef48cde" (UID: "fb436a2b-e048-4e84-ba17-24a49ef48cde"). InnerVolumeSpecName "kube-api-access-4spsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:21:47.306435 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.306400    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb436a2b-e048-4e84-ba17-24a49ef48cde-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fb436a2b-e048-4e84-ba17-24a49ef48cde" (UID: "fb436a2b-e048-4e84-ba17-24a49ef48cde"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:21:47.405199 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.405164    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4spsc\" (UniqueName: \"kubernetes.io/projected/fb436a2b-e048-4e84-ba17-24a49ef48cde-kube-api-access-4spsc\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:21:47.405199 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.405196    2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb436a2b-e048-4e84-ba17-24a49ef48cde-webhook-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:21:47.421809 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.421780    2557 generic.go:356] "Generic (PLEG): container finished" podID="fb436a2b-e048-4e84-ba17-24a49ef48cde" containerID="19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c" exitCode=0
Jul 29 00:21:47.421942 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.421827    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9bf644fc-skpl7"
Jul 29 00:21:47.421942 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.421870    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9bf644fc-skpl7" event={"ID":"fb436a2b-e048-4e84-ba17-24a49ef48cde","Type":"ContainerDied","Data":"19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c"}
Jul 29 00:21:47.421942 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.421926    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9bf644fc-skpl7" event={"ID":"fb436a2b-e048-4e84-ba17-24a49ef48cde","Type":"ContainerDied","Data":"d908bf995a3b01f7e78d99ad4a744c579e43592de0adb42e030aa2ad1fbeaacc"}
Jul 29 00:21:47.422171 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.421955    2557 scope.go:117] "RemoveContainer" containerID="19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c"
Jul 29 00:21:47.432991 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.432976    2557 scope.go:117] "RemoveContainer" containerID="19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c"
Jul 29 00:21:47.433238 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:21:47.433218    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c\": container with ID starting with 19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c not found: ID does not exist" containerID="19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c"
Jul 29 00:21:47.433286 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.433248    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c"} err="failed to get container status \"19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c\": rpc error: code = NotFound desc = could not find container \"19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c\": container with ID starting with 19b931ae6aeffbb74652a232abf1ccebd7d55c38c28e984500e410c72ebcc37c not found: ID does not exist"
Jul 29 00:21:47.445403 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.445383    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-skpl7"]
Jul 29 00:21:47.449693 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.449673    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-skpl7"]
Jul 29 00:21:47.940371 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:47.940332    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb436a2b-e048-4e84-ba17-24a49ef48cde" path="/var/lib/kubelet/pods/fb436a2b-e048-4e84-ba17-24a49ef48cde/volumes"
Jul 29 00:21:48.226645 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.226568    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f9bf644fc-tp8r8"]
Jul 29 00:21:48.227002 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.226903    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb436a2b-e048-4e84-ba17-24a49ef48cde" containerName="manager"
Jul 29 00:21:48.227002 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.226915    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb436a2b-e048-4e84-ba17-24a49ef48cde" containerName="manager"
Jul 29 00:21:48.227002 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.226988    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb436a2b-e048-4e84-ba17-24a49ef48cde" containerName="manager"
Jul 29 00:21:48.231399 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.231376    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9bf644fc-tp8r8"
Jul 29 00:21:48.233700 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.233682    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vdjvz\""
Jul 29 00:21:48.233700 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.233692    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 00:21:48.233828 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.233682    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 00:21:48.239691 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.239645    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-tp8r8"]
Jul 29 00:21:48.314543 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.314510    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96c5fa1d-10b9-4b8d-8af5-3155173f9e44-webhook-cert\") pod \"maas-controller-f9bf644fc-tp8r8\" (UID: \"96c5fa1d-10b9-4b8d-8af5-3155173f9e44\") " pod="opendatahub/maas-controller-f9bf644fc-tp8r8"
Jul 29 00:21:48.314702 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.314578    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmqlt\" (UniqueName: \"kubernetes.io/projected/96c5fa1d-10b9-4b8d-8af5-3155173f9e44-kube-api-access-mmqlt\") pod \"maas-controller-f9bf644fc-tp8r8\" (UID: \"96c5fa1d-10b9-4b8d-8af5-3155173f9e44\") " pod="opendatahub/maas-controller-f9bf644fc-tp8r8"
Jul 29 00:21:48.415443 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.415417    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmqlt\" (UniqueName: \"kubernetes.io/projected/96c5fa1d-10b9-4b8d-8af5-3155173f9e44-kube-api-access-mmqlt\") pod \"maas-controller-f9bf644fc-tp8r8\" (UID: \"96c5fa1d-10b9-4b8d-8af5-3155173f9e44\") " pod="opendatahub/maas-controller-f9bf644fc-tp8r8"
Jul 29 00:21:48.415575 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.415483    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96c5fa1d-10b9-4b8d-8af5-3155173f9e44-webhook-cert\") pod \"maas-controller-f9bf644fc-tp8r8\" (UID: \"96c5fa1d-10b9-4b8d-8af5-3155173f9e44\") " pod="opendatahub/maas-controller-f9bf644fc-tp8r8"
Jul 29 00:21:48.417850 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.417817    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96c5fa1d-10b9-4b8d-8af5-3155173f9e44-webhook-cert\") pod \"maas-controller-f9bf644fc-tp8r8\" (UID: \"96c5fa1d-10b9-4b8d-8af5-3155173f9e44\") " pod="opendatahub/maas-controller-f9bf644fc-tp8r8"
Jul 29 00:21:48.423211 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.423185    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmqlt\" (UniqueName: \"kubernetes.io/projected/96c5fa1d-10b9-4b8d-8af5-3155173f9e44-kube-api-access-mmqlt\") pod \"maas-controller-f9bf644fc-tp8r8\" (UID: \"96c5fa1d-10b9-4b8d-8af5-3155173f9e44\") " pod="opendatahub/maas-controller-f9bf644fc-tp8r8"
Jul 29 00:21:48.542911 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.542833    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9bf644fc-tp8r8"
Jul 29 00:21:48.669170 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:48.669147    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f9bf644fc-tp8r8"]
Jul 29 00:21:48.671384 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:21:48.671358    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c5fa1d_10b9_4b8d_8af5_3155173f9e44.slice/crio-2445d30d7bf463b56ba8f892037e305fa7ba96f779ce36ee0f238203233dce6b WatchSource:0}: Error finding container 2445d30d7bf463b56ba8f892037e305fa7ba96f779ce36ee0f238203233dce6b: Status 404 returned error can't find the container with id 2445d30d7bf463b56ba8f892037e305fa7ba96f779ce36ee0f238203233dce6b
Jul 29 00:21:49.433855 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:49.433770    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9bf644fc-tp8r8" event={"ID":"96c5fa1d-10b9-4b8d-8af5-3155173f9e44","Type":"ContainerStarted","Data":"8cf22aa67c8f62ce712f3f0c1175bd7414e738a772fb4fae9194e1ea06a0b200"}
Jul 29 00:21:49.433855 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:49.433823    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9bf644fc-tp8r8" event={"ID":"96c5fa1d-10b9-4b8d-8af5-3155173f9e44","Type":"ContainerStarted","Data":"2445d30d7bf463b56ba8f892037e305fa7ba96f779ce36ee0f238203233dce6b"}
Jul 29 00:21:49.434231 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:49.433903    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f9bf644fc-tp8r8"
Jul 29 00:21:49.449013 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:21:49.448963    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f9bf644fc-tp8r8" podStartSLOduration=1.001806048 podStartE2EDuration="1.448943631s" totalImagesPullingTime="447.137583ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:21:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:21:49.448045387 +0000 UTC m=+824.026742831" watchObservedRunningTime="2026-07-29 00:21:49.448943631 +0000 UTC m=+824.027641076"
Jul 29 00:22:00.440893 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:22:00.440862    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f9bf644fc-tp8r8"
Jul 29 00:23:05.815119 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:23:05.815029    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:23:05.815634 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:23:05.815438    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:28:05.842330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:28:05.842293    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:28:05.844897 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:28:05.843013    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:33:05.869818 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:33:05.869786    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:33:05.872589 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:33:05.872568    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:35:47.636152 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.636105    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"]
Jul 29 00:35:47.639445 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.639423    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:47.641302 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.641282    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4b0b4c35-serving-cert\""
Jul 29 00:35:47.650347 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.650324    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"]
Jul 29 00:35:47.785794 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.785759    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47c5133-76a2-4db5-a396-7b32d5736ae7-openshift-service-ca\") pod \"maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:47.785963 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.785814    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47c5133-76a2-4db5-a396-7b32d5736ae7-maas-api-tls\") pod \"maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:47.785963 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.785873    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq8gv\" (UniqueName: \"kubernetes.io/projected/f47c5133-76a2-4db5-a396-7b32d5736ae7-kube-api-access-gq8gv\") pod \"maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:47.887024 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.886937    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47c5133-76a2-4db5-a396-7b32d5736ae7-openshift-service-ca\") pod \"maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:47.887024 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.886994    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47c5133-76a2-4db5-a396-7b32d5736ae7-maas-api-tls\") pod \"maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:47.887024 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.887025    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8gv\" (UniqueName: \"kubernetes.io/projected/f47c5133-76a2-4db5-a396-7b32d5736ae7-kube-api-access-gq8gv\") pod \"maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:47.887755 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.887728    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47c5133-76a2-4db5-a396-7b32d5736ae7-openshift-service-ca\") pod \"maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:47.889606 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.889582    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47c5133-76a2-4db5-a396-7b32d5736ae7-maas-api-tls\") pod \"maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:47.895428 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.895401    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq8gv\" (UniqueName: \"kubernetes.io/projected/f47c5133-76a2-4db5-a396-7b32d5736ae7-kube-api-access-gq8gv\") pod \"maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:47.951403 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:47.951376    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:48.092169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.092144    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"]
Jul 29 00:35:48.095458 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:35:48.094913    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47c5133_76a2_4db5_a396_7b32d5736ae7.slice/crio-ece2a07a250170a303df5643fd23ddc9a68c28773bfae725833ab60498b811cf WatchSource:0}: Error finding container ece2a07a250170a303df5643fd23ddc9a68c28773bfae725833ab60498b811cf: Status 404 returned error can't find the container with id ece2a07a250170a303df5643fd23ddc9a68c28773bfae725833ab60498b811cf
Jul 29 00:35:48.096784 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.096749    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:35:48.155619 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.155536    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc" event={"ID":"f47c5133-76a2-4db5-a396-7b32d5736ae7","Type":"ContainerStarted","Data":"ece2a07a250170a303df5643fd23ddc9a68c28773bfae725833ab60498b811cf"}
Jul 29 00:35:48.458810 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.458722    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"]
Jul 29 00:35:48.462891 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.462873    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:35:48.466781 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.466477    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-4b0b4c35\""
Jul 29 00:35:48.466781 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.466534    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-4b0b4c35-dockercfg-qbrdp\""
Jul 29 00:35:48.476547 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.476508    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"]
Jul 29 00:35:48.530453 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.530410    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"]
Jul 29 00:35:48.533864 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.533841    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:35:48.544508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.544476    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"]
Jul 29 00:35:48.593054 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.593015    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6\" (UID: \"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:35:48.593211 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.593077    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqxbk\" (UniqueName: \"kubernetes.io/projected/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-kube-api-access-nqxbk\") pod \"payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6\" (UID: \"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:35:48.693980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.693937    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g58sx\" (UniqueName: \"kubernetes.io/projected/56627d18-ec52-4e0e-aef7-b8948d69d7ea-kube-api-access-g58sx\") pod \"payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx\" (UID: \"56627d18-ec52-4e0e-aef7-b8948d69d7ea\") " pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:35:48.694393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.694097    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6\" (UID: \"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:35:48.694393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.694168    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56627d18-ec52-4e0e-aef7-b8948d69d7ea-plugins-config-volume\") pod \"payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx\" (UID: \"56627d18-ec52-4e0e-aef7-b8948d69d7ea\") " pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:35:48.694393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.694205    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqxbk\" (UniqueName: \"kubernetes.io/projected/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-kube-api-access-nqxbk\") pod \"payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6\" (UID: \"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:35:48.695016 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.694991    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6\" (UID: \"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:35:48.709047 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.708987    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqxbk\" (UniqueName: \"kubernetes.io/projected/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-kube-api-access-nqxbk\") pod \"payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6\" (UID: \"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:35:48.781502 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.781468    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:35:48.795794 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.795767    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g58sx\" (UniqueName: \"kubernetes.io/projected/56627d18-ec52-4e0e-aef7-b8948d69d7ea-kube-api-access-g58sx\") pod \"payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx\" (UID: \"56627d18-ec52-4e0e-aef7-b8948d69d7ea\") " pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:35:48.795938 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.795875    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56627d18-ec52-4e0e-aef7-b8948d69d7ea-plugins-config-volume\") pod \"payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx\" (UID: \"56627d18-ec52-4e0e-aef7-b8948d69d7ea\") " pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:35:48.797041 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.797020    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56627d18-ec52-4e0e-aef7-b8948d69d7ea-plugins-config-volume\") pod \"payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx\" (UID: \"56627d18-ec52-4e0e-aef7-b8948d69d7ea\") " pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:35:48.804753 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.804705    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g58sx\" (UniqueName: \"kubernetes.io/projected/56627d18-ec52-4e0e-aef7-b8948d69d7ea-kube-api-access-g58sx\") pod \"payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx\" (UID: \"56627d18-ec52-4e0e-aef7-b8948d69d7ea\") " pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:35:48.845912 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.845869    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:35:48.939209 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:48.939183    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"]
Jul 29 00:35:48.941024 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:35:48.940997    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf8db2e_7dc8_4fc7_ab1b_2ccbad819153.slice/crio-2894fd2a6416b7d9e90ceebd13687d1f6449ba81b89171a3efc3336eb0ae16bc WatchSource:0}: Error finding container 2894fd2a6416b7d9e90ceebd13687d1f6449ba81b89171a3efc3336eb0ae16bc: Status 404 returned error can't find the container with id 2894fd2a6416b7d9e90ceebd13687d1f6449ba81b89171a3efc3336eb0ae16bc
Jul 29 00:35:49.001203 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.001177    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"]
Jul 29 00:35:49.003150 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:35:49.003126    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56627d18_ec52_4e0e_aef7_b8948d69d7ea.slice/crio-b50d6be37cbd76d76f16af263bb1284af2ed96e1ed77b0537759282a7a352f19 WatchSource:0}: Error finding container b50d6be37cbd76d76f16af263bb1284af2ed96e1ed77b0537759282a7a352f19: Status 404 returned error can't find the container with id b50d6be37cbd76d76f16af263bb1284af2ed96e1ed77b0537759282a7a352f19
Jul 29 00:35:49.162418 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.162376    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc" event={"ID":"f47c5133-76a2-4db5-a396-7b32d5736ae7","Type":"ContainerStarted","Data":"f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4"}
Jul 29 00:35:49.162622 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.162578    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:35:49.164304 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.164273    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx" event={"ID":"56627d18-ec52-4e0e-aef7-b8948d69d7ea","Type":"ContainerStarted","Data":"a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0"}
Jul 29 00:35:49.164619 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.164335    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:35:49.164619 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.164365    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx" event={"ID":"56627d18-ec52-4e0e-aef7-b8948d69d7ea","Type":"ContainerStarted","Data":"b50d6be37cbd76d76f16af263bb1284af2ed96e1ed77b0537759282a7a352f19"}
Jul 29 00:35:49.166110 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.166086    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6" event={"ID":"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153","Type":"ContainerStarted","Data":"e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca"}
Jul 29 00:35:49.166203 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.166127    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6" event={"ID":"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153","Type":"ContainerStarted","Data":"2894fd2a6416b7d9e90ceebd13687d1f6449ba81b89171a3efc3336eb0ae16bc"}
Jul 29 00:35:49.166249 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.166232    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:35:49.189503 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.189447    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc" podStartSLOduration=1.629415191 podStartE2EDuration="2.189426166s" totalImagesPullingTime="560.010975ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:35:49.189284338 +0000 UTC m=+1663.767981787" watchObservedRunningTime="2026-07-29 00:35:49.189426166 +0000 UTC m=+1663.768123612"
Jul 29 00:35:49.210811 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.210714    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx" podStartSLOduration=1.210695748 podStartE2EDuration="1.210695748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:35:49.20936596 +0000 UTC m=+1663.788063404" watchObservedRunningTime="2026-07-29 00:35:49.210695748 +0000 UTC m=+1663.789393193"
Jul 29 00:35:49.227748 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:49.227647    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6" podStartSLOduration=1.227627222 podStartE2EDuration="1.227627222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:35:49.226848355 +0000 UTC m=+1663.805545801" watchObservedRunningTime="2026-07-29 00:35:49.227627222 +0000 UTC m=+1663.806324669"
Jul 29 00:35:55.176674 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:35:55.176631    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:36:00.172471 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:00.172435    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:36:00.172895 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:00.172784    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:36:18.215253 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.215218    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"]
Jul 29 00:36:18.215819 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.215452    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc" podUID="f47c5133-76a2-4db5-a396-7b32d5736ae7" containerName="maas-api" containerID="cri-o://f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4" gracePeriod=30
Jul 29 00:36:18.275058 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.275013    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"]
Jul 29 00:36:18.275351 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.275289    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx" podUID="56627d18-ec52-4e0e-aef7-b8948d69d7ea" containerName="payload-processing" containerID="cri-o://a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0" gracePeriod=30
Jul 29 00:36:18.283258 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.283218    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"]
Jul 29 00:36:18.283724 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.283436    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6" podUID="ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153" containerName="payload-pre-processing" containerID="cri-o://e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca" gracePeriod=30
Jul 29 00:36:18.550987 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.550960    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:36:18.589165 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.589138    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:36:18.604286 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.604260    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:36:18.670858 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.670832    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47c5133-76a2-4db5-a396-7b32d5736ae7-maas-api-tls\") pod \"f47c5133-76a2-4db5-a396-7b32d5736ae7\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") "
Jul 29 00:36:18.671040 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.670935    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-plugins-config-volume\") pod \"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153\" (UID: \"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153\") "
Jul 29 00:36:18.671040 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.670967    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56627d18-ec52-4e0e-aef7-b8948d69d7ea-plugins-config-volume\") pod \"56627d18-ec52-4e0e-aef7-b8948d69d7ea\" (UID: \"56627d18-ec52-4e0e-aef7-b8948d69d7ea\") "
Jul 29 00:36:18.671040 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.671009    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqxbk\" (UniqueName: \"kubernetes.io/projected/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-kube-api-access-nqxbk\") pod \"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153\" (UID: \"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153\") "
Jul 29 00:36:18.671040 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.671036    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gq8gv\" (UniqueName: \"kubernetes.io/projected/f47c5133-76a2-4db5-a396-7b32d5736ae7-kube-api-access-gq8gv\") pod \"f47c5133-76a2-4db5-a396-7b32d5736ae7\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") "
Jul 29 00:36:18.671248 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.671078    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g58sx\" (UniqueName: \"kubernetes.io/projected/56627d18-ec52-4e0e-aef7-b8948d69d7ea-kube-api-access-g58sx\") pod \"56627d18-ec52-4e0e-aef7-b8948d69d7ea\" (UID: \"56627d18-ec52-4e0e-aef7-b8948d69d7ea\") "
Jul 29 00:36:18.671248 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.671142    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47c5133-76a2-4db5-a396-7b32d5736ae7-openshift-service-ca\") pod \"f47c5133-76a2-4db5-a396-7b32d5736ae7\" (UID: \"f47c5133-76a2-4db5-a396-7b32d5736ae7\") "
Jul 29 00:36:18.671418 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.671346    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153" (UID: "ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:36:18.671502 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.671401    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56627d18-ec52-4e0e-aef7-b8948d69d7ea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56627d18-ec52-4e0e-aef7-b8948d69d7ea" (UID: "56627d18-ec52-4e0e-aef7-b8948d69d7ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:36:18.671502 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.671482    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:36:18.671734 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.671710    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f47c5133-76a2-4db5-a396-7b32d5736ae7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f47c5133-76a2-4db5-a396-7b32d5736ae7" (UID: "f47c5133-76a2-4db5-a396-7b32d5736ae7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:36:18.673238 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.673208    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56627d18-ec52-4e0e-aef7-b8948d69d7ea-kube-api-access-g58sx" (OuterVolumeSpecName: "kube-api-access-g58sx") pod "56627d18-ec52-4e0e-aef7-b8948d69d7ea" (UID: "56627d18-ec52-4e0e-aef7-b8948d69d7ea"). InnerVolumeSpecName "kube-api-access-g58sx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:36:18.673332 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.673246    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-kube-api-access-nqxbk" (OuterVolumeSpecName: "kube-api-access-nqxbk") pod "ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153" (UID: "ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153"). InnerVolumeSpecName "kube-api-access-nqxbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:36:18.673457 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.673436    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f47c5133-76a2-4db5-a396-7b32d5736ae7-kube-api-access-gq8gv" (OuterVolumeSpecName: "kube-api-access-gq8gv") pod "f47c5133-76a2-4db5-a396-7b32d5736ae7" (UID: "f47c5133-76a2-4db5-a396-7b32d5736ae7"). InnerVolumeSpecName "kube-api-access-gq8gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:36:18.673519 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.673439    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f47c5133-76a2-4db5-a396-7b32d5736ae7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f47c5133-76a2-4db5-a396-7b32d5736ae7" (UID: "f47c5133-76a2-4db5-a396-7b32d5736ae7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:36:18.771961 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.771884    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47c5133-76a2-4db5-a396-7b32d5736ae7-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:36:18.771961 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.771909    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f47c5133-76a2-4db5-a396-7b32d5736ae7-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:36:18.771961 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.771919    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56627d18-ec52-4e0e-aef7-b8948d69d7ea-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:36:18.771961 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.771930    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqxbk\" (UniqueName: \"kubernetes.io/projected/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153-kube-api-access-nqxbk\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:36:18.771961 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.771939    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gq8gv\" (UniqueName: \"kubernetes.io/projected/f47c5133-76a2-4db5-a396-7b32d5736ae7-kube-api-access-gq8gv\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:36:18.771961 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:18.771948    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g58sx\" (UniqueName: \"kubernetes.io/projected/56627d18-ec52-4e0e-aef7-b8948d69d7ea-kube-api-access-g58sx\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:36:19.310786 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.310747    2557 generic.go:356] "Generic (PLEG): container finished" podID="56627d18-ec52-4e0e-aef7-b8948d69d7ea" containerID="a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0" exitCode=0
Jul 29 00:36:19.311199 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.310808    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"
Jul 29 00:36:19.311199 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.310825    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx" event={"ID":"56627d18-ec52-4e0e-aef7-b8948d69d7ea","Type":"ContainerDied","Data":"a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0"}
Jul 29 00:36:19.311199 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.310875    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx" event={"ID":"56627d18-ec52-4e0e-aef7-b8948d69d7ea","Type":"ContainerDied","Data":"b50d6be37cbd76d76f16af263bb1284af2ed96e1ed77b0537759282a7a352f19"}
Jul 29 00:36:19.311199 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.310896    2557 scope.go:117] "RemoveContainer" containerID="a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0"
Jul 29 00:36:19.312239 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.312215    2557 generic.go:356] "Generic (PLEG): container finished" podID="ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153" containerID="e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca" exitCode=0
Jul 29 00:36:19.312362 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.312247    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6" event={"ID":"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153","Type":"ContainerDied","Data":"e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca"}
Jul 29 00:36:19.312362 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.312271    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"
Jul 29 00:36:19.312362 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.312298    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6" event={"ID":"ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153","Type":"ContainerDied","Data":"2894fd2a6416b7d9e90ceebd13687d1f6449ba81b89171a3efc3336eb0ae16bc"}
Jul 29 00:36:19.313725 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.313705    2557 generic.go:356] "Generic (PLEG): container finished" podID="f47c5133-76a2-4db5-a396-7b32d5736ae7" containerID="f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4" exitCode=0
Jul 29 00:36:19.313820 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.313763    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"
Jul 29 00:36:19.313820 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.313783    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc" event={"ID":"f47c5133-76a2-4db5-a396-7b32d5736ae7","Type":"ContainerDied","Data":"f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4"}
Jul 29 00:36:19.313920 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.313824    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc" event={"ID":"f47c5133-76a2-4db5-a396-7b32d5736ae7","Type":"ContainerDied","Data":"ece2a07a250170a303df5643fd23ddc9a68c28773bfae725833ab60498b811cf"}
Jul 29 00:36:19.320243 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.320222    2557 scope.go:117] "RemoveContainer" containerID="a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0"
Jul 29 00:36:19.320598 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:36:19.320510    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0\": container with ID starting with a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0 not found: ID does not exist" containerID="a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0"
Jul 29 00:36:19.320598 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.320539    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0"} err="failed to get container status \"a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0\": rpc error: code = NotFound desc = could not find container \"a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0\": container with ID starting with a9cc79ae802293e941a2cf47e19129591abc0a3f5fac6b9963346166813f69b0 not found: ID does not exist"
Jul 29 00:36:19.320598 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.320555    2557 scope.go:117] "RemoveContainer" containerID="e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca"
Jul 29 00:36:19.329902 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.329885    2557 scope.go:117] "RemoveContainer" containerID="e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca"
Jul 29 00:36:19.330152 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:36:19.330133    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca\": container with ID starting with e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca not found: ID does not exist" containerID="e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca"
Jul 29 00:36:19.330206 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.330159    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca"} err="failed to get container status \"e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca\": rpc error: code = NotFound desc = could not find container \"e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca\": container with ID starting with e0e005848117e5e4f3f3a3672a6dc940b4097669ebad5bf13f6b5969eeeda8ca not found: ID does not exist"
Jul 29 00:36:19.330206 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.330175    2557 scope.go:117] "RemoveContainer" containerID="f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4"
Jul 29 00:36:19.337030 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.337009    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"]
Jul 29 00:36:19.337518 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.337507    2557 scope.go:117] "RemoveContainer" containerID="f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4"
Jul 29 00:36:19.337756 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:36:19.337739    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4\": container with ID starting with f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4 not found: ID does not exist" containerID="f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4"
Jul 29 00:36:19.337812 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.337760    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4"} err="failed to get container status \"f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4\": rpc error: code = NotFound desc = could not find container \"f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4\": container with ID starting with f11b9b278a53ba53c5d232d00edd858dcf697536b0185db18c11c618372375f4 not found: ID does not exist"
Jul 29 00:36:19.351347 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.351328    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4b0b4c35-78b8768dfd-fngtx"]
Jul 29 00:36:19.365064 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.365045    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"]
Jul 29 00:36:19.367744 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.367724    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4b0b4c35-66cbd4458d-gldb6"]
Jul 29 00:36:19.381367 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.381344    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"]
Jul 29 00:36:19.384988 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.384968    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b0b4c35-5867bfd4f5-dmnrc"]
Jul 29 00:36:19.940935 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.940908    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56627d18-ec52-4e0e-aef7-b8948d69d7ea" path="/var/lib/kubelet/pods/56627d18-ec52-4e0e-aef7-b8948d69d7ea/volumes"
Jul 29 00:36:19.941308 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.941294    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153" path="/var/lib/kubelet/pods/ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153/volumes"
Jul 29 00:36:19.941639 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:19.941628    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f47c5133-76a2-4db5-a396-7b32d5736ae7" path="/var/lib/kubelet/pods/f47c5133-76a2-4db5-a396-7b32d5736ae7/volumes"
Jul 29 00:36:21.970899 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.970854    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"]
Jul 29 00:36:21.971344 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.971263    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f47c5133-76a2-4db5-a396-7b32d5736ae7" containerName="maas-api"
Jul 29 00:36:21.971344 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.971280    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f47c5133-76a2-4db5-a396-7b32d5736ae7" containerName="maas-api"
Jul 29 00:36:21.971344 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.971295    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56627d18-ec52-4e0e-aef7-b8948d69d7ea" containerName="payload-processing"
Jul 29 00:36:21.971344 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.971303    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="56627d18-ec52-4e0e-aef7-b8948d69d7ea" containerName="payload-processing"
Jul 29 00:36:21.971344 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.971325    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153" containerName="payload-pre-processing"
Jul 29 00:36:21.971344 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.971336    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153" containerName="payload-pre-processing"
Jul 29 00:36:21.971565 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.971430    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="56627d18-ec52-4e0e-aef7-b8948d69d7ea" containerName="payload-processing"
Jul 29 00:36:21.971565 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.971444    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f47c5133-76a2-4db5-a396-7b32d5736ae7" containerName="maas-api"
Jul 29 00:36:21.971565 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.971453    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecf8db2e-7dc8-4fc7-ab1b-2ccbad819153" containerName="payload-pre-processing"
Jul 29 00:36:21.974683 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.974648    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:21.979209 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.979163    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-1abe847c-serving-cert\""
Jul 29 00:36:21.989556 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:21.989536    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"]
Jul 29 00:36:22.103697 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.103652    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggpmf\" (UniqueName: \"kubernetes.io/projected/abbf4677-9d82-4f5a-a723-d65520310f36-kube-api-access-ggpmf\") pod \"maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:22.103858 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.103745    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abbf4677-9d82-4f5a-a723-d65520310f36-maas-api-tls\") pod \"maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:22.103858 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.103770    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abbf4677-9d82-4f5a-a723-d65520310f36-openshift-service-ca\") pod \"maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:22.204453 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.204420    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abbf4677-9d82-4f5a-a723-d65520310f36-maas-api-tls\") pod \"maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:22.204621 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.204478    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abbf4677-9d82-4f5a-a723-d65520310f36-openshift-service-ca\") pod \"maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:22.204621 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.204522    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggpmf\" (UniqueName: \"kubernetes.io/projected/abbf4677-9d82-4f5a-a723-d65520310f36-kube-api-access-ggpmf\") pod \"maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:22.205183 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.205161    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abbf4677-9d82-4f5a-a723-d65520310f36-openshift-service-ca\") pod \"maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:22.206932 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.206910    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abbf4677-9d82-4f5a-a723-d65520310f36-maas-api-tls\") pod \"maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:22.215230 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.215206    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggpmf\" (UniqueName: \"kubernetes.io/projected/abbf4677-9d82-4f5a-a723-d65520310f36-kube-api-access-ggpmf\") pod \"maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:22.286232 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.286149    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:22.434289 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.434264    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"]
Jul 29 00:36:22.436695 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:36:22.436638    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabbf4677_9d82_4f5a_a723_d65520310f36.slice/crio-e3c3065b40d174795cadb642a5f4d2e0c6377165aa46663d10ffd47c8740d09a WatchSource:0}: Error finding container e3c3065b40d174795cadb642a5f4d2e0c6377165aa46663d10ffd47c8740d09a: Status 404 returned error can't find the container with id e3c3065b40d174795cadb642a5f4d2e0c6377165aa46663d10ffd47c8740d09a
Jul 29 00:36:22.727304 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.727271    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"]
Jul 29 00:36:22.730560 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.730542    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:36:22.733471 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.733447    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-1abe847c\""
Jul 29 00:36:22.747642 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.747620    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"]
Jul 29 00:36:22.760542 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.760521    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"]
Jul 29 00:36:22.763647 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.763632    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:36:22.766617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.766600    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-1abe847c-dockercfg-qgsrg\""
Jul 29 00:36:22.789207 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.789187    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"]
Jul 29 00:36:22.809874 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.809852    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl\" (UID: \"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:36:22.809962 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.809884    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfkcc\" (UniqueName: \"kubernetes.io/projected/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-kube-api-access-mfkcc\") pod \"payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl\" (UID: \"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:36:22.911030 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.910998    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-299jv\" (UniqueName: \"kubernetes.io/projected/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-kube-api-access-299jv\") pod \"payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n\" (UID: \"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662\") " pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:36:22.911198 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.911045    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n\" (UID: \"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662\") " pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:36:22.911198 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.911168    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl\" (UID: \"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:36:22.911273 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.911206    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfkcc\" (UniqueName: \"kubernetes.io/projected/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-kube-api-access-mfkcc\") pod \"payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl\" (UID: \"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:36:22.911920 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.911901    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl\" (UID: \"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:36:22.922406 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:22.922384    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfkcc\" (UniqueName: \"kubernetes.io/projected/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-kube-api-access-mfkcc\") pod \"payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl\" (UID: \"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:36:23.012011 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.011986    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-299jv\" (UniqueName: \"kubernetes.io/projected/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-kube-api-access-299jv\") pod \"payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n\" (UID: \"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662\") " pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:36:23.012371 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.012024    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n\" (UID: \"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662\") " pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:36:23.012732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.012707    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n\" (UID: \"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662\") " pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:36:23.020354 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.020330    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-299jv\" (UniqueName: \"kubernetes.io/projected/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-kube-api-access-299jv\") pod \"payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n\" (UID: \"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662\") " pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:36:23.040896 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.040875    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:36:23.073569 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.073540    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:36:23.250336 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.250118    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"]
Jul 29 00:36:23.253063 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:36:23.253036    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5ed9f0_eb3d_44f7_b09a_db49aae3ce95.slice/crio-9f8e4f569815d74e34c594fc1754d169d9d7b495c6df4b46f27ff9499489007d WatchSource:0}: Error finding container 9f8e4f569815d74e34c594fc1754d169d9d7b495c6df4b46f27ff9499489007d: Status 404 returned error can't find the container with id 9f8e4f569815d74e34c594fc1754d169d9d7b495c6df4b46f27ff9499489007d
Jul 29 00:36:23.262208 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.262176    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"]
Jul 29 00:36:23.264447 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:36:23.264425    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc9f8a4c_7e0d_457b_8f99_8f4b63e6c662.slice/crio-3b103e853de937bc46ff2df48119b599f187e13dea61d204cff54fed129e3f87 WatchSource:0}: Error finding container 3b103e853de937bc46ff2df48119b599f187e13dea61d204cff54fed129e3f87: Status 404 returned error can't find the container with id 3b103e853de937bc46ff2df48119b599f187e13dea61d204cff54fed129e3f87
Jul 29 00:36:23.336904 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.336879    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q" event={"ID":"abbf4677-9d82-4f5a-a723-d65520310f36","Type":"ContainerStarted","Data":"5d05d214c3cddf924ccc00f3f02819a5ffdd20a29e1b72e6bf5ffaf498eff208"}
Jul 29 00:36:23.337012 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.336926    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q" event={"ID":"abbf4677-9d82-4f5a-a723-d65520310f36","Type":"ContainerStarted","Data":"e3c3065b40d174795cadb642a5f4d2e0c6377165aa46663d10ffd47c8740d09a"}
Jul 29 00:36:23.337080 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.337032    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:23.338612 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.338582    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n" event={"ID":"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662","Type":"ContainerStarted","Data":"44bace8d7f1bec0f67b597390caf616b175295830ba42acdfa0a4a3bffb4b5d2"}
Jul 29 00:36:23.338731 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.338626    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n" event={"ID":"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662","Type":"ContainerStarted","Data":"3b103e853de937bc46ff2df48119b599f187e13dea61d204cff54fed129e3f87"}
Jul 29 00:36:23.338731 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.338723    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:36:23.340107 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.340087    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl" event={"ID":"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95","Type":"ContainerStarted","Data":"713c53239715ae8d9a58c601f7c82ec0e54c3245ac0e18d29b6c3d8c998ce402"}
Jul 29 00:36:23.340209 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.340127    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl" event={"ID":"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95","Type":"ContainerStarted","Data":"9f8e4f569815d74e34c594fc1754d169d9d7b495c6df4b46f27ff9499489007d"}
Jul 29 00:36:23.340209 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.340167    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:36:23.362160 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.362103    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q" podStartSLOduration=1.802649437 podStartE2EDuration="2.362088226s" totalImagesPullingTime="559.438789ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:23.360291199 +0000 UTC m=+1697.938988647" watchObservedRunningTime="2026-07-29 00:36:23.362088226 +0000 UTC m=+1697.940785671"
Jul 29 00:36:23.377679 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.377607    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl" podStartSLOduration=1.377587868 podStartE2EDuration="1.377587868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:23.376635143 +0000 UTC m=+1697.955332588" watchObservedRunningTime="2026-07-29 00:36:23.377587868 +0000 UTC m=+1697.956285314"
Jul 29 00:36:23.395074 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:23.395035    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n" podStartSLOduration=1.395021414 podStartE2EDuration="1.395021414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:36:23.392537503 +0000 UTC m=+1697.971234949" watchObservedRunningTime="2026-07-29 00:36:23.395021414 +0000 UTC m=+1697.973718858"
Jul 29 00:36:29.351121 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:29.351040    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:36:34.346424 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:34.346396    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:36:34.346925 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:36:34.346737    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:37:03.322836 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.322784    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"]
Jul 29 00:37:03.323409 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.323036    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q" podUID="abbf4677-9d82-4f5a-a723-d65520310f36" containerName="maas-api" containerID="cri-o://5d05d214c3cddf924ccc00f3f02819a5ffdd20a29e1b72e6bf5ffaf498eff208" gracePeriod=30
Jul 29 00:37:03.389858 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.389824    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"]
Jul 29 00:37:03.390090 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.390061    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n" podUID="dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662" containerName="payload-processing" containerID="cri-o://44bace8d7f1bec0f67b597390caf616b175295830ba42acdfa0a4a3bffb4b5d2" gracePeriod=30
Jul 29 00:37:03.395467 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.395437    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"]
Jul 29 00:37:03.395735 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.395706    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl" podUID="ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95" containerName="payload-pre-processing" containerID="cri-o://713c53239715ae8d9a58c601f7c82ec0e54c3245ac0e18d29b6c3d8c998ce402" gracePeriod=30
Jul 29 00:37:03.525175 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.525141    2557 generic.go:356] "Generic (PLEG): container finished" podID="abbf4677-9d82-4f5a-a723-d65520310f36" containerID="5d05d214c3cddf924ccc00f3f02819a5ffdd20a29e1b72e6bf5ffaf498eff208" exitCode=0
Jul 29 00:37:03.525320 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.525297    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q" event={"ID":"abbf4677-9d82-4f5a-a723-d65520310f36","Type":"ContainerDied","Data":"5d05d214c3cddf924ccc00f3f02819a5ffdd20a29e1b72e6bf5ffaf498eff208"}
Jul 29 00:37:03.527137 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.527090    2557 generic.go:356] "Generic (PLEG): container finished" podID="dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662" containerID="44bace8d7f1bec0f67b597390caf616b175295830ba42acdfa0a4a3bffb4b5d2" exitCode=0
Jul 29 00:37:03.527254 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.527208    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n" event={"ID":"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662","Type":"ContainerDied","Data":"44bace8d7f1bec0f67b597390caf616b175295830ba42acdfa0a4a3bffb4b5d2"}
Jul 29 00:37:03.529412 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.529354    2557 generic.go:356] "Generic (PLEG): container finished" podID="ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95" containerID="713c53239715ae8d9a58c601f7c82ec0e54c3245ac0e18d29b6c3d8c998ce402" exitCode=0
Jul 29 00:37:03.529412 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.529405    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl" event={"ID":"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95","Type":"ContainerDied","Data":"713c53239715ae8d9a58c601f7c82ec0e54c3245ac0e18d29b6c3d8c998ce402"}
Jul 29 00:37:03.663263 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.663239    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:37:03.667518 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.667495    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:37:03.711710 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.711686    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:37:03.766598 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.766562    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-plugins-config-volume\") pod \"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662\" (UID: \"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662\") "
Jul 29 00:37:03.766835 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.766610    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-299jv\" (UniqueName: \"kubernetes.io/projected/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-kube-api-access-299jv\") pod \"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662\" (UID: \"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662\") "
Jul 29 00:37:03.766835 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.766774    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggpmf\" (UniqueName: \"kubernetes.io/projected/abbf4677-9d82-4f5a-a723-d65520310f36-kube-api-access-ggpmf\") pod \"abbf4677-9d82-4f5a-a723-d65520310f36\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") "
Jul 29 00:37:03.766835 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.766806    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfkcc\" (UniqueName: \"kubernetes.io/projected/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-kube-api-access-mfkcc\") pod \"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95\" (UID: \"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95\") "
Jul 29 00:37:03.767017 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.766850    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abbf4677-9d82-4f5a-a723-d65520310f36-maas-api-tls\") pod \"abbf4677-9d82-4f5a-a723-d65520310f36\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") "
Jul 29 00:37:03.767017 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.766980    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-plugins-config-volume\") pod \"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95\" (UID: \"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95\") "
Jul 29 00:37:03.767126 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.767009    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662" (UID: "dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:37:03.767126 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.767064    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abbf4677-9d82-4f5a-a723-d65520310f36-openshift-service-ca\") pod \"abbf4677-9d82-4f5a-a723-d65520310f36\" (UID: \"abbf4677-9d82-4f5a-a723-d65520310f36\") "
Jul 29 00:37:03.767358 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.767321    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95" (UID: "ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:37:03.767429 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.767367    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:03.767587 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.767563    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abbf4677-9d82-4f5a-a723-d65520310f36-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "abbf4677-9d82-4f5a-a723-d65520310f36" (UID: "abbf4677-9d82-4f5a-a723-d65520310f36"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:37:03.768848 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.768812    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-kube-api-access-299jv" (OuterVolumeSpecName: "kube-api-access-299jv") pod "dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662" (UID: "dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662"). InnerVolumeSpecName "kube-api-access-299jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:37:03.768967 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.768855    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abbf4677-9d82-4f5a-a723-d65520310f36-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "abbf4677-9d82-4f5a-a723-d65520310f36" (UID: "abbf4677-9d82-4f5a-a723-d65520310f36"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:37:03.769024 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.768962    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-kube-api-access-mfkcc" (OuterVolumeSpecName: "kube-api-access-mfkcc") pod "ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95" (UID: "ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95"). InnerVolumeSpecName "kube-api-access-mfkcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:37:03.769194 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.769178    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abbf4677-9d82-4f5a-a723-d65520310f36-kube-api-access-ggpmf" (OuterVolumeSpecName: "kube-api-access-ggpmf") pod "abbf4677-9d82-4f5a-a723-d65520310f36" (UID: "abbf4677-9d82-4f5a-a723-d65520310f36"). InnerVolumeSpecName "kube-api-access-ggpmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:37:03.868060 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.868026    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abbf4677-9d82-4f5a-a723-d65520310f36-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:03.868060 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.868054    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:03.868060 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.868064    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abbf4677-9d82-4f5a-a723-d65520310f36-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:03.868285 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.868073    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-299jv\" (UniqueName: \"kubernetes.io/projected/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662-kube-api-access-299jv\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:03.868285 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.868083    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggpmf\" (UniqueName: \"kubernetes.io/projected/abbf4677-9d82-4f5a-a723-d65520310f36-kube-api-access-ggpmf\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:03.868285 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:03.868092    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfkcc\" (UniqueName: \"kubernetes.io/projected/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95-kube-api-access-mfkcc\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:04.482950 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.482917    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"]
Jul 29 00:37:04.483336 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.483322    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662" containerName="payload-processing"
Jul 29 00:37:04.483400 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.483338    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662" containerName="payload-processing"
Jul 29 00:37:04.483400 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.483352    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abbf4677-9d82-4f5a-a723-d65520310f36" containerName="maas-api"
Jul 29 00:37:04.483400 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.483358    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="abbf4677-9d82-4f5a-a723-d65520310f36" containerName="maas-api"
Jul 29 00:37:04.483400 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.483374    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95" containerName="payload-pre-processing"
Jul 29 00:37:04.483400 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.483381    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95" containerName="payload-pre-processing"
Jul 29 00:37:04.483617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.483443    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662" containerName="payload-processing"
Jul 29 00:37:04.483617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.483456    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="abbf4677-9d82-4f5a-a723-d65520310f36" containerName="maas-api"
Jul 29 00:37:04.483617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.483468    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95" containerName="payload-pre-processing"
Jul 29 00:37:04.486995 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.486971    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.488803 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.488784    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a969ca8e-openshift-default-dockercfg-zfkm2\""
Jul 29 00:37:04.501020 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.500996    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"]
Jul 29 00:37:04.535413 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.535377    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n" event={"ID":"dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662","Type":"ContainerDied","Data":"3b103e853de937bc46ff2df48119b599f187e13dea61d204cff54fed129e3f87"}
Jul 29 00:37:04.535548 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.535389    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"
Jul 29 00:37:04.535548 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.535432    2557 scope.go:117] "RemoveContainer" containerID="44bace8d7f1bec0f67b597390caf616b175295830ba42acdfa0a4a3bffb4b5d2"
Jul 29 00:37:04.536777 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.536760    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"
Jul 29 00:37:04.536866 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.536792    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl" event={"ID":"ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95","Type":"ContainerDied","Data":"9f8e4f569815d74e34c594fc1754d169d9d7b495c6df4b46f27ff9499489007d"}
Jul 29 00:37:04.538635 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.538613    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q" event={"ID":"abbf4677-9d82-4f5a-a723-d65520310f36","Type":"ContainerDied","Data":"e3c3065b40d174795cadb642a5f4d2e0c6377165aa46663d10ffd47c8740d09a"}
Jul 29 00:37:04.538755 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.538729    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"
Jul 29 00:37:04.545338 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.545318    2557 scope.go:117] "RemoveContainer" containerID="713c53239715ae8d9a58c601f7c82ec0e54c3245ac0e18d29b6c3d8c998ce402"
Jul 29 00:37:04.553754 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.553736    2557 scope.go:117] "RemoveContainer" containerID="5d05d214c3cddf924ccc00f3f02819a5ffdd20a29e1b72e6bf5ffaf498eff208"
Jul 29 00:37:04.555928 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.555910    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"]
Jul 29 00:37:04.558933 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.558912    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1abe847c-58b9bc74df-qrc9n"]
Jul 29 00:37:04.567712 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.567691    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"]
Jul 29 00:37:04.573842 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.573822    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1abe847c-6fcfbc67b-56m2q"]
Jul 29 00:37:04.579728 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.579701    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vl92\" (UniqueName: \"kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-kube-api-access-2vl92\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.579841 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.579754    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-data\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.579902 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.579837    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-envoy\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.579902 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.579862    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-podinfo\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.580048 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.579911    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-token\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.580048 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.579948    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-socket\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.580048 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.579975    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-credential-socket\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.580048 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.580014    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istiod-ca-cert\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.580245 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.580128    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-certs\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.585298 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.585275    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"]
Jul 29 00:37:04.593134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.593064    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1abe847c-597c7476bd-9rxpl"]
Jul 29 00:37:04.680613 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.680573    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istiod-ca-cert\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.680816 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.680637    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-certs\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.680816 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.680697    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vl92\" (UniqueName: \"kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-kube-api-access-2vl92\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.680816 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.680721    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-data\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.680816 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.680761    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-envoy\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.680816 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.680780    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-podinfo\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.681089 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.680825    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-token\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.681089 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.680853    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-socket\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.681089 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.680878    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-credential-socket\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.681238 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.681088    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-certs\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.681238 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.681216    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-data\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.681471 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.681450    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-socket\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.681539 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.681473    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-credential-socket\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.681539 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.681502    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istiod-ca-cert\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.683029 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.683006    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-envoy\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.683788 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.683769    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-podinfo\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.689798 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.689779    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-token\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.690130 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.690107    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vl92\" (UniqueName: \"kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-kube-api-access-2vl92\") pod \"e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") " pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.797755 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.797726    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:04.936042 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.936016    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"]
Jul 29 00:37:04.937973 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:37:04.937946    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7becbe3_0fb7_4165_84d9_8c38d897d5ea.slice/crio-6eca253f9e2a85b6d05f14804216c52363f35df18cae88cec93427fd21f5a5da WatchSource:0}: Error finding container 6eca253f9e2a85b6d05f14804216c52363f35df18cae88cec93427fd21f5a5da: Status 404 returned error can't find the container with id 6eca253f9e2a85b6d05f14804216c52363f35df18cae88cec93427fd21f5a5da
Jul 29 00:37:04.939957 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.939929    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:37:04.940012 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.939990    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:37:04.940053 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:04.940017    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:37:05.548934 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:05.548901    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" event={"ID":"d7becbe3-0fb7-4165-84d9-8c38d897d5ea","Type":"ContainerStarted","Data":"5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774"}
Jul 29 00:37:05.549297 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:05.548951    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" event={"ID":"d7becbe3-0fb7-4165-84d9-8c38d897d5ea","Type":"ContainerStarted","Data":"6eca253f9e2a85b6d05f14804216c52363f35df18cae88cec93427fd21f5a5da"}
Jul 29 00:37:05.569869 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:05.569824    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" podStartSLOduration=1.5698101169999998 podStartE2EDuration="1.569810117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:37:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:05.569443756 +0000 UTC m=+1740.148141201" watchObservedRunningTime="2026-07-29 00:37:05.569810117 +0000 UTC m=+1740.148507561"
Jul 29 00:37:05.798732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:05.798702    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:05.942123 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:05.942094    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abbf4677-9d82-4f5a-a723-d65520310f36" path="/var/lib/kubelet/pods/abbf4677-9d82-4f5a-a723-d65520310f36/volumes"
Jul 29 00:37:05.942636 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:05.942613    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662" path="/var/lib/kubelet/pods/dc9f8a4c-7e0d-457b-8f99-8f4b63e6c662/volumes"
Jul 29 00:37:05.943134 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:05.943117    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95" path="/var/lib/kubelet/pods/ef5ed9f0-eb3d-44f7-b09a-db49aae3ce95/volumes"
Jul 29 00:37:06.803952 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:06.803920    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:07.049541 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.049509    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"]
Jul 29 00:37:07.055163 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.055091    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.057379 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.057354    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a969ca8e-serving-cert\""
Jul 29 00:37:07.065734 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.065712    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"]
Jul 29 00:37:07.104937 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.104903    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-openshift-service-ca\") pod \"maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.105074 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.104942    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmkd\" (UniqueName: \"kubernetes.io/projected/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-kube-api-access-cbmkd\") pod \"maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.105074 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.105032    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-maas-api-tls\") pod \"maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.206191 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.206161    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-openshift-service-ca\") pod \"maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.206375 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.206203    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmkd\" (UniqueName: \"kubernetes.io/projected/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-kube-api-access-cbmkd\") pod \"maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.206375 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.206254    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-maas-api-tls\") pod \"maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.206474 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:37:07.206377    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-serving-cert: secret "maas-api-e2e-ait-a969ca8e-serving-cert" not found
Jul 29 00:37:07.206474 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:37:07.206453    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-maas-api-tls podName:474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1 nodeName:}" failed. No retries permitted until 2026-07-29 00:37:07.706428992 +0000 UTC m=+1742.285126414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-maas-api-tls") pod "maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8" (UID: "474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1") : secret "maas-api-e2e-ait-a969ca8e-serving-cert" not found
Jul 29 00:37:07.207052 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.207027    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-openshift-service-ca\") pod \"maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.220592 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.220561    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmkd\" (UniqueName: \"kubernetes.io/projected/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-kube-api-access-cbmkd\") pod \"maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.558710 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.558673    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:07.559560 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.559540    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:07.711197 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.711161    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-maas-api-tls\") pod \"maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.713567 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.713536    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-maas-api-tls\") pod \"maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.907606 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.907574    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"]
Jul 29 00:37:07.911002 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.910986    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:07.912800 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.912773    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a969ca8e\""
Jul 29 00:37:07.912906 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.912849    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a969ca8e-dockercfg-rt4p8\""
Jul 29 00:37:07.921809 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.921787    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"]
Jul 29 00:37:07.967191 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.967163    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:07.981137 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.981112    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"]
Jul 29 00:37:07.985156 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.985134    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:07.994017 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:07.993994    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"]
Jul 29 00:37:08.020190 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.020155    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vlff\" (UniqueName: \"kubernetes.io/projected/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-kube-api-access-6vlff\") pod \"payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4\" (UID: \"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:08.020386 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.020280    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4\" (UID: \"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:08.121799 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.121758    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jl2t\" (UniqueName: \"kubernetes.io/projected/48180bc1-cc62-4c33-9585-b7030fb10a11-kube-api-access-5jl2t\") pod \"payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz\" (UID: \"48180bc1-cc62-4c33-9585-b7030fb10a11\") " pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:08.121799 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.121794    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48180bc1-cc62-4c33-9585-b7030fb10a11-plugins-config-volume\") pod \"payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz\" (UID: \"48180bc1-cc62-4c33-9585-b7030fb10a11\") " pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:08.122031 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.121827    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vlff\" (UniqueName: \"kubernetes.io/projected/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-kube-api-access-6vlff\") pod \"payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4\" (UID: \"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:08.122031 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.121937    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4\" (UID: \"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:08.122636 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.122610    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4\" (UID: \"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:08.132917 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.132895    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vlff\" (UniqueName: \"kubernetes.io/projected/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-kube-api-access-6vlff\") pod \"payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4\" (UID: \"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:08.222598 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.222512    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:08.222852 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.222829    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jl2t\" (UniqueName: \"kubernetes.io/projected/48180bc1-cc62-4c33-9585-b7030fb10a11-kube-api-access-5jl2t\") pod \"payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz\" (UID: \"48180bc1-cc62-4c33-9585-b7030fb10a11\") " pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:08.222912 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.222864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48180bc1-cc62-4c33-9585-b7030fb10a11-plugins-config-volume\") pod \"payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz\" (UID: \"48180bc1-cc62-4c33-9585-b7030fb10a11\") " pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:08.223558 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.223534    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48180bc1-cc62-4c33-9585-b7030fb10a11-plugins-config-volume\") pod \"payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz\" (UID: \"48180bc1-cc62-4c33-9585-b7030fb10a11\") " pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:08.236312 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.236281    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jl2t\" (UniqueName: \"kubernetes.io/projected/48180bc1-cc62-4c33-9585-b7030fb10a11-kube-api-access-5jl2t\") pod \"payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz\" (UID: \"48180bc1-cc62-4c33-9585-b7030fb10a11\") " pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:08.301323 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.301292    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:08.339203 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.339176    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"]
Jul 29 00:37:08.341793 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:37:08.341638    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod474b02f4_0f1c_4fbc_ad2e_b3e5ef0529c1.slice/crio-da7b00a1ed163f78e4c1bea8a11afe37883fc2d20902d7f488ab1a6fdf2c2f50 WatchSource:0}: Error finding container da7b00a1ed163f78e4c1bea8a11afe37883fc2d20902d7f488ab1a6fdf2c2f50: Status 404 returned error can't find the container with id da7b00a1ed163f78e4c1bea8a11afe37883fc2d20902d7f488ab1a6fdf2c2f50
Jul 29 00:37:08.369169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.368649    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"]
Jul 29 00:37:08.371799 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:37:08.371768    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9338de_43b7_4aff_a3a9_5a7e0059a9d4.slice/crio-8bc48774f36881df8dc2d7132c551008141554af5567fe3c460fcceb75989dbe WatchSource:0}: Error finding container 8bc48774f36881df8dc2d7132c551008141554af5567fe3c460fcceb75989dbe: Status 404 returned error can't find the container with id 8bc48774f36881df8dc2d7132c551008141554af5567fe3c460fcceb75989dbe
Jul 29 00:37:08.450970 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.450940    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"]
Jul 29 00:37:08.453382 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:37:08.453350    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48180bc1_cc62_4c33_9585_b7030fb10a11.slice/crio-e4251e357bed3ec303e72b4038e7d7f9f28234744ee99d69556c2138538d5412 WatchSource:0}: Error finding container e4251e357bed3ec303e72b4038e7d7f9f28234744ee99d69556c2138538d5412: Status 404 returned error can't find the container with id e4251e357bed3ec303e72b4038e7d7f9f28234744ee99d69556c2138538d5412
Jul 29 00:37:08.564508 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.564472    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz" event={"ID":"48180bc1-cc62-4c33-9585-b7030fb10a11","Type":"ContainerStarted","Data":"2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4"}
Jul 29 00:37:08.564713 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.564533    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz" event={"ID":"48180bc1-cc62-4c33-9585-b7030fb10a11","Type":"ContainerStarted","Data":"e4251e357bed3ec303e72b4038e7d7f9f28234744ee99d69556c2138538d5412"}
Jul 29 00:37:08.564713 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.564691    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:08.566095 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.566063    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4" event={"ID":"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4","Type":"ContainerStarted","Data":"02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185"}
Jul 29 00:37:08.566211 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.566112    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4" event={"ID":"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4","Type":"ContainerStarted","Data":"8bc48774f36881df8dc2d7132c551008141554af5567fe3c460fcceb75989dbe"}
Jul 29 00:37:08.566211 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.566143    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:08.567223 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.567200    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8" event={"ID":"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1","Type":"ContainerStarted","Data":"da7b00a1ed163f78e4c1bea8a11afe37883fc2d20902d7f488ab1a6fdf2c2f50"}
Jul 29 00:37:08.584734 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.584687    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz" podStartSLOduration=1.5846738299999998 podStartE2EDuration="1.58467383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:37:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:08.582483644 +0000 UTC m=+1743.161181088" watchObservedRunningTime="2026-07-29 00:37:08.58467383 +0000 UTC m=+1743.163371269"
Jul 29 00:37:08.600982 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:08.600939    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4" podStartSLOduration=1.60092022 podStartE2EDuration="1.60092022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:37:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:08.59927975 +0000 UTC m=+1743.177977200" watchObservedRunningTime="2026-07-29 00:37:08.60092022 +0000 UTC m=+1743.179617666"
Jul 29 00:37:09.575262 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:09.575224    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8" event={"ID":"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1","Type":"ContainerStarted","Data":"4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2"}
Jul 29 00:37:09.575724 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:09.575392    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:09.592769 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:09.592717    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8" podStartSLOduration=2.141610556 podStartE2EDuration="2.592701651s" totalImagesPullingTime="451.091095ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:09.590586454 +0000 UTC m=+1744.169283904" watchObservedRunningTime="2026-07-29 00:37:09.592701651 +0000 UTC m=+1744.171399095"
Jul 29 00:37:15.585727 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:15.585697    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:19.577565 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:19.577529    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:19.577963 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:19.577623    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:21.637118 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:21.637087    2557 generic.go:356] "Generic (PLEG): container finished" podID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerID="5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774" exitCode=0
Jul 29 00:37:21.637537 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:21.637159    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" event={"ID":"d7becbe3-0fb7-4165-84d9-8c38d897d5ea","Type":"ContainerDied","Data":"5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774"}
Jul 29 00:37:21.637580 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:21.637562    2557 scope.go:117] "RemoveContainer" containerID="5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774"
Jul 29 00:37:21.638641 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:21.638613    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:37:21.638727 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:21.638702    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:37:21.638769 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:21.638735    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:37:22.558718 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:22.558681    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:22.643484 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:22.643447    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" event={"ID":"d7becbe3-0fb7-4165-84d9-8c38d897d5ea","Type":"ContainerStarted","Data":"6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06"}
Jul 29 00:37:22.798523 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:22.798494    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:22.799696 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:22.799647    2557 patch_prober.go:28] interesting pod/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 29 00:37:22.799828 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:22.799712    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 29 00:37:23.798823 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:23.798787    2557 patch_prober.go:28] interesting pod/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 29 00:37:23.799204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:23.798842    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 29 00:37:24.798406 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:24.798368    2557 patch_prober.go:28] interesting pod/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 29 00:37:24.798584 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:24.798434    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 29 00:37:25.798236 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:25.798201    2557 patch_prober.go:28] interesting pod/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 29 00:37:25.798648 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:25.798260    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 29 00:37:26.802398 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:26.802364    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:26.802817 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:26.802773    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:26.803390 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:26.803373    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:37.920844 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:37.920810    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"]
Jul 29 00:37:37.921342 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:37.921078    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8" podUID="474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1" containerName="maas-api" containerID="cri-o://4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2" gracePeriod=30
Jul 29 00:37:37.982738 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:37.982703    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"]
Jul 29 00:37:37.983029 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:37.982991    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz" podUID="48180bc1-cc62-4c33-9585-b7030fb10a11" containerName="payload-processing" containerID="cri-o://2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4" gracePeriod=30
Jul 29 00:37:37.992112 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:37.992080    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"]
Jul 29 00:37:37.992349 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:37.992296    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4" podUID="7c9338de-43b7-4aff-a3a9-5a7e0059a9d4" containerName="payload-pre-processing" containerID="cri-o://02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185" gracePeriod=30
Jul 29 00:37:38.276049 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.275886    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:38.283820 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.283799    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:38.305022 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.304997    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbmkd\" (UniqueName: \"kubernetes.io/projected/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-kube-api-access-cbmkd\") pod \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") "
Jul 29 00:37:38.305187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.305035    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jl2t\" (UniqueName: \"kubernetes.io/projected/48180bc1-cc62-4c33-9585-b7030fb10a11-kube-api-access-5jl2t\") pod \"48180bc1-cc62-4c33-9585-b7030fb10a11\" (UID: \"48180bc1-cc62-4c33-9585-b7030fb10a11\") "
Jul 29 00:37:38.305187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.305058    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-maas-api-tls\") pod \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") "
Jul 29 00:37:38.305187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.305159    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-openshift-service-ca\") pod \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\" (UID: \"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1\") "
Jul 29 00:37:38.305187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.305184    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48180bc1-cc62-4c33-9585-b7030fb10a11-plugins-config-volume\") pod \"48180bc1-cc62-4c33-9585-b7030fb10a11\" (UID: \"48180bc1-cc62-4c33-9585-b7030fb10a11\") "
Jul 29 00:37:38.305558 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.305509    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1" (UID: "474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:37:38.305694 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.305675    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48180bc1-cc62-4c33-9585-b7030fb10a11-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "48180bc1-cc62-4c33-9585-b7030fb10a11" (UID: "48180bc1-cc62-4c33-9585-b7030fb10a11"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:37:38.307527 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.307502    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-kube-api-access-cbmkd" (OuterVolumeSpecName: "kube-api-access-cbmkd") pod "474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1" (UID: "474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1"). InnerVolumeSpecName "kube-api-access-cbmkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:37:38.308040 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.308018    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48180bc1-cc62-4c33-9585-b7030fb10a11-kube-api-access-5jl2t" (OuterVolumeSpecName: "kube-api-access-5jl2t") pod "48180bc1-cc62-4c33-9585-b7030fb10a11" (UID: "48180bc1-cc62-4c33-9585-b7030fb10a11"). InnerVolumeSpecName "kube-api-access-5jl2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:37:38.308167 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.308145    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1" (UID: "474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:37:38.324023 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.323992    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:38.406715 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.406680    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vlff\" (UniqueName: \"kubernetes.io/projected/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-kube-api-access-6vlff\") pod \"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4\" (UID: \"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4\") "
Jul 29 00:37:38.406899 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.406765    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-plugins-config-volume\") pod \"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4\" (UID: \"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4\") "
Jul 29 00:37:38.406973 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.406959    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:38.407036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.406974    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48180bc1-cc62-4c33-9585-b7030fb10a11-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:38.407036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.406988    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbmkd\" (UniqueName: \"kubernetes.io/projected/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-kube-api-access-cbmkd\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:38.407036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.406999    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jl2t\" (UniqueName: \"kubernetes.io/projected/48180bc1-cc62-4c33-9585-b7030fb10a11-kube-api-access-5jl2t\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:38.407036 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.407010    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:38.407176 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.407094    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c9338de-43b7-4aff-a3a9-5a7e0059a9d4" (UID: "7c9338de-43b7-4aff-a3a9-5a7e0059a9d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:37:38.408646 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.408624    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-kube-api-access-6vlff" (OuterVolumeSpecName: "kube-api-access-6vlff") pod "7c9338de-43b7-4aff-a3a9-5a7e0059a9d4" (UID: "7c9338de-43b7-4aff-a3a9-5a7e0059a9d4"). InnerVolumeSpecName "kube-api-access-6vlff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:37:38.508525 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.508452    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:38.508525 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.508479    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vlff\" (UniqueName: \"kubernetes.io/projected/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4-kube-api-access-6vlff\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:38.739017 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.738979    2557 generic.go:356] "Generic (PLEG): container finished" podID="474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1" containerID="4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2" exitCode=0
Jul 29 00:37:38.739190 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.739025    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"
Jul 29 00:37:38.739190 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.739070    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8" event={"ID":"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1","Type":"ContainerDied","Data":"4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2"}
Jul 29 00:37:38.739190 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.739127    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8" event={"ID":"474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1","Type":"ContainerDied","Data":"da7b00a1ed163f78e4c1bea8a11afe37883fc2d20902d7f488ab1a6fdf2c2f50"}
Jul 29 00:37:38.739190 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.739158    2557 scope.go:117] "RemoveContainer" containerID="4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2"
Jul 29 00:37:38.740498 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.740471    2557 generic.go:356] "Generic (PLEG): container finished" podID="48180bc1-cc62-4c33-9585-b7030fb10a11" containerID="2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4" exitCode=0
Jul 29 00:37:38.740633 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.740510    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"
Jul 29 00:37:38.740633 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.740532    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz" event={"ID":"48180bc1-cc62-4c33-9585-b7030fb10a11","Type":"ContainerDied","Data":"2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4"}
Jul 29 00:37:38.740633 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.740597    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz" event={"ID":"48180bc1-cc62-4c33-9585-b7030fb10a11","Type":"ContainerDied","Data":"e4251e357bed3ec303e72b4038e7d7f9f28234744ee99d69556c2138538d5412"}
Jul 29 00:37:38.742127 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.742067    2557 generic.go:356] "Generic (PLEG): container finished" podID="7c9338de-43b7-4aff-a3a9-5a7e0059a9d4" containerID="02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185" exitCode=0
Jul 29 00:37:38.742127 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.742102    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4" event={"ID":"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4","Type":"ContainerDied","Data":"02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185"}
Jul 29 00:37:38.742339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.742162    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4" event={"ID":"7c9338de-43b7-4aff-a3a9-5a7e0059a9d4","Type":"ContainerDied","Data":"8bc48774f36881df8dc2d7132c551008141554af5567fe3c460fcceb75989dbe"}
Jul 29 00:37:38.742339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.742115    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"
Jul 29 00:37:38.749612 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.749590    2557 scope.go:117] "RemoveContainer" containerID="4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2"
Jul 29 00:37:38.750398 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:37:38.750365    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2\": container with ID starting with 4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2 not found: ID does not exist" containerID="4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2"
Jul 29 00:37:38.750534 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.750407    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2"} err="failed to get container status \"4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2\": rpc error: code = NotFound desc = could not find container \"4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2\": container with ID starting with 4e5247f93bf14229a0d9e19f2def01def976f190dddc71db50822e3c3d7711c2 not found: ID does not exist"
Jul 29 00:37:38.750534 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.750445    2557 scope.go:117] "RemoveContainer" containerID="2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4"
Jul 29 00:37:38.759055 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.759025    2557 scope.go:117] "RemoveContainer" containerID="2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4"
Jul 29 00:37:38.759269 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:37:38.759249    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4\": container with ID starting with 2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4 not found: ID does not exist" containerID="2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4"
Jul 29 00:37:38.759316 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.759275    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4"} err="failed to get container status \"2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4\": rpc error: code = NotFound desc = could not find container \"2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4\": container with ID starting with 2d1e0147fddd922af35d35db02adccbfb7066e22803e4233778ef81404a333a4 not found: ID does not exist"
Jul 29 00:37:38.759316 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.759292    2557 scope.go:117] "RemoveContainer" containerID="02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185"
Jul 29 00:37:38.766628 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.766601    2557 scope.go:117] "RemoveContainer" containerID="02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185"
Jul 29 00:37:38.766888 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:37:38.766865    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185\": container with ID starting with 02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185 not found: ID does not exist" containerID="02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185"
Jul 29 00:37:38.766948 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.766896    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185"} err="failed to get container status \"02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185\": rpc error: code = NotFound desc = could not find container \"02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185\": container with ID starting with 02159ce10e317a7141270eb6c6371c05d0745d5fa3aad4a2acb4801d1450e185 not found: ID does not exist"
Jul 29 00:37:38.767970 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.767954    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"]
Jul 29 00:37:38.772108 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.772086    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a969ca8e-67df67949b-mlzrz"]
Jul 29 00:37:38.779885 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.779866    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"]
Jul 29 00:37:38.784247 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.784228    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a969ca8e-58676768d6-tfqn8"]
Jul 29 00:37:38.792703 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.792683    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"]
Jul 29 00:37:38.799321 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:38.799301    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a969ca8e-c76ffb7dd-mjhq4"]
Jul 29 00:37:39.293907 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:39.293872    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"]
Jul 29 00:37:39.294384 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:39.294068    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy" containerID="cri-o://6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06" gracePeriod=30
Jul 29 00:37:39.940322 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:39.940290    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1" path="/var/lib/kubelet/pods/474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1/volumes"
Jul 29 00:37:39.940652 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:39.940638    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48180bc1-cc62-4c33-9585-b7030fb10a11" path="/var/lib/kubelet/pods/48180bc1-cc62-4c33-9585-b7030fb10a11/volumes"
Jul 29 00:37:39.940976 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:39.940964    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c9338de-43b7-4aff-a3a9-5a7e0059a9d4" path="/var/lib/kubelet/pods/7c9338de-43b7-4aff-a3a9-5a7e0059a9d4/volumes"
Jul 29 00:37:41.804823 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:41.804783    2557 patch_prober.go:28] interesting pod/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 29 00:37:41.805193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:41.804852    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 00:37:42.149184 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.149145    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"]
Jul 29 00:37:42.149679 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.149648    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1" containerName="maas-api"
Jul 29 00:37:42.149679 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.149680    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1" containerName="maas-api"
Jul 29 00:37:42.149808 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.149709    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c9338de-43b7-4aff-a3a9-5a7e0059a9d4" containerName="payload-pre-processing"
Jul 29 00:37:42.149808 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.149718    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c9338de-43b7-4aff-a3a9-5a7e0059a9d4" containerName="payload-pre-processing"
Jul 29 00:37:42.149808 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.149739    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48180bc1-cc62-4c33-9585-b7030fb10a11" containerName="payload-processing"
Jul 29 00:37:42.149808 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.149748    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="48180bc1-cc62-4c33-9585-b7030fb10a11" containerName="payload-processing"
Jul 29 00:37:42.149934 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.149832    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="474b02f4-0f1c-4fbc-ad2e-b3e5ef0529c1" containerName="maas-api"
Jul 29 00:37:42.149934 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.149846    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c9338de-43b7-4aff-a3a9-5a7e0059a9d4" containerName="payload-pre-processing"
Jul 29 00:37:42.149934 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.149857    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="48180bc1-cc62-4c33-9585-b7030fb10a11" containerName="payload-processing"
Jul 29 00:37:42.153248 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.153227    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.155490 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.155466    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a0eed2d7-serving-cert\""
Jul 29 00:37:42.163335 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.163313    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"]
Jul 29 00:37:42.241302 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.241257    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-openshift-service-ca\") pod \"maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.241489 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.241371    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-maas-api-tls\") pod \"maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.241489 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.241400    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wglqk\" (UniqueName: \"kubernetes.io/projected/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-kube-api-access-wglqk\") pod \"maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.342208 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.342179    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-openshift-service-ca\") pod \"maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.342369 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.342241    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-maas-api-tls\") pod \"maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.342369 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.342271    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wglqk\" (UniqueName: \"kubernetes.io/projected/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-kube-api-access-wglqk\") pod \"maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.342454 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:37:42.342400    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-serving-cert: secret "maas-api-e2e-derive-a0eed2d7-serving-cert" not found
Jul 29 00:37:42.342488 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:37:42.342470    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-maas-api-tls podName:0ae3c7bf-ee97-4985-9cca-b693d61fa7bb nodeName:}" failed. No retries permitted until 2026-07-29 00:37:42.842452061 +0000 UTC m=+1777.421149484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-maas-api-tls") pod "maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6" (UID: "0ae3c7bf-ee97-4985-9cca-b693d61fa7bb") : secret "maas-api-e2e-derive-a0eed2d7-serving-cert" not found
Jul 29 00:37:42.342919 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.342900    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-openshift-service-ca\") pod \"maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.360159 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.360127    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wglqk\" (UniqueName: \"kubernetes.io/projected/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-kube-api-access-wglqk\") pod \"maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.846265 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.846229    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-maas-api-tls\") pod \"maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.848567 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.848543    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-maas-api-tls\") pod \"maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:42.906103 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.906051    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"]
Jul 29 00:37:42.908447 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.908425    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:37:42.910911 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.910891    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-a0eed2d7-dockercfg-9q228\""
Jul 29 00:37:42.911028 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.910894    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-a0eed2d7\""
Jul 29 00:37:42.922636 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.922611    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"]
Jul 29 00:37:42.935764 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.935738    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"]
Jul 29 00:37:42.938129 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.938108    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:37:42.947393 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.947365    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/324ae4dd-72ed-47ac-ac28-0c7039bffb46-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8\" (UID: \"324ae4dd-72ed-47ac-ac28-0c7039bffb46\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:37:42.947475 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.947430    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9zb6\" (UniqueName: \"kubernetes.io/projected/324ae4dd-72ed-47ac-ac28-0c7039bffb46-kube-api-access-t9zb6\") pod \"payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8\" (UID: \"324ae4dd-72ed-47ac-ac28-0c7039bffb46\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:37:42.954555 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:42.954533    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"]
Jul 29 00:37:43.048436 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.048402    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54b62\" (UniqueName: \"kubernetes.io/projected/63bde123-fa4f-45fc-9132-dfe83a48fcc0-kube-api-access-54b62\") pod \"payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf\" (UID: \"63bde123-fa4f-45fc-9132-dfe83a48fcc0\") " pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:37:43.048629 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.048470    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/324ae4dd-72ed-47ac-ac28-0c7039bffb46-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8\" (UID: \"324ae4dd-72ed-47ac-ac28-0c7039bffb46\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:37:43.048629 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.048613    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9zb6\" (UniqueName: \"kubernetes.io/projected/324ae4dd-72ed-47ac-ac28-0c7039bffb46-kube-api-access-t9zb6\") pod \"payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8\" (UID: \"324ae4dd-72ed-47ac-ac28-0c7039bffb46\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:37:43.048766 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.048671    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63bde123-fa4f-45fc-9132-dfe83a48fcc0-plugins-config-volume\") pod \"payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf\" (UID: \"63bde123-fa4f-45fc-9132-dfe83a48fcc0\") " pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:37:43.049306 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.049283    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/324ae4dd-72ed-47ac-ac28-0c7039bffb46-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8\" (UID: \"324ae4dd-72ed-47ac-ac28-0c7039bffb46\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:37:43.057845 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.057808    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9zb6\" (UniqueName: \"kubernetes.io/projected/324ae4dd-72ed-47ac-ac28-0c7039bffb46-kube-api-access-t9zb6\") pod \"payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8\" (UID: \"324ae4dd-72ed-47ac-ac28-0c7039bffb46\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:37:43.064392 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.064370    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:43.150148 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.150102    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63bde123-fa4f-45fc-9132-dfe83a48fcc0-plugins-config-volume\") pod \"payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf\" (UID: \"63bde123-fa4f-45fc-9132-dfe83a48fcc0\") " pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:37:43.150322 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.150169    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54b62\" (UniqueName: \"kubernetes.io/projected/63bde123-fa4f-45fc-9132-dfe83a48fcc0-kube-api-access-54b62\") pod \"payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf\" (UID: \"63bde123-fa4f-45fc-9132-dfe83a48fcc0\") " pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:37:43.151403 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.151349    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63bde123-fa4f-45fc-9132-dfe83a48fcc0-plugins-config-volume\") pod \"payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf\" (UID: \"63bde123-fa4f-45fc-9132-dfe83a48fcc0\") " pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:37:43.159939 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.159909    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54b62\" (UniqueName: \"kubernetes.io/projected/63bde123-fa4f-45fc-9132-dfe83a48fcc0-kube-api-access-54b62\") pod \"payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf\" (UID: \"63bde123-fa4f-45fc-9132-dfe83a48fcc0\") " pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:37:43.210732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.210679    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"]
Jul 29 00:37:43.212871 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:37:43.212845    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae3c7bf_ee97_4985_9cca_b693d61fa7bb.slice/crio-2f84fd0ef9c8512b6aa2733b79d37f4039e23d9d1efa588d93c8d7f460dec552 WatchSource:0}: Error finding container 2f84fd0ef9c8512b6aa2733b79d37f4039e23d9d1efa588d93c8d7f460dec552: Status 404 returned error can't find the container with id 2f84fd0ef9c8512b6aa2733b79d37f4039e23d9d1efa588d93c8d7f460dec552
Jul 29 00:37:43.218780 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.218759    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:37:43.247433 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.247408    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:37:43.383060 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.383034    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"]
Jul 29 00:37:43.385948 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:37:43.385916    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod324ae4dd_72ed_47ac_ac28_0c7039bffb46.slice/crio-c981f0cb57f9a7e4d082965efd2fea9cc17f14ae8af4ac062d81d9103cafa5e7 WatchSource:0}: Error finding container c981f0cb57f9a7e4d082965efd2fea9cc17f14ae8af4ac062d81d9103cafa5e7: Status 404 returned error can't find the container with id c981f0cb57f9a7e4d082965efd2fea9cc17f14ae8af4ac062d81d9103cafa5e7
Jul 29 00:37:43.601680 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.601641    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"]
Jul 29 00:37:43.603181 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:37:43.603155    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63bde123_fa4f_45fc_9132_dfe83a48fcc0.slice/crio-a09c0c13ba120a4ac97fae9c07e64ef43680ca2c0c223c1178561747609603b1 WatchSource:0}: Error finding container a09c0c13ba120a4ac97fae9c07e64ef43680ca2c0c223c1178561747609603b1: Status 404 returned error can't find the container with id a09c0c13ba120a4ac97fae9c07e64ef43680ca2c0c223c1178561747609603b1
Jul 29 00:37:43.773494 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.773453    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8" event={"ID":"324ae4dd-72ed-47ac-ac28-0c7039bffb46","Type":"ContainerStarted","Data":"5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a"}
Jul 29 00:37:43.773704 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.773525    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8" event={"ID":"324ae4dd-72ed-47ac-ac28-0c7039bffb46","Type":"ContainerStarted","Data":"c981f0cb57f9a7e4d082965efd2fea9cc17f14ae8af4ac062d81d9103cafa5e7"}
Jul 29 00:37:43.773704 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.773553    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:37:43.775547 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.775503    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf" event={"ID":"63bde123-fa4f-45fc-9132-dfe83a48fcc0","Type":"ContainerStarted","Data":"50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a"}
Jul 29 00:37:43.775712 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.775557    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf" event={"ID":"63bde123-fa4f-45fc-9132-dfe83a48fcc0","Type":"ContainerStarted","Data":"a09c0c13ba120a4ac97fae9c07e64ef43680ca2c0c223c1178561747609603b1"}
Jul 29 00:37:43.775712 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.775590    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:37:43.777417 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.777393    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6" event={"ID":"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb","Type":"ContainerStarted","Data":"97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e"}
Jul 29 00:37:43.777551 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.777433    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6" event={"ID":"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb","Type":"ContainerStarted","Data":"2f84fd0ef9c8512b6aa2733b79d37f4039e23d9d1efa588d93c8d7f460dec552"}
Jul 29 00:37:43.777551 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.777522    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:43.790842 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.790800    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8" podStartSLOduration=1.7907862429999999 podStartE2EDuration="1.790786243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:37:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:43.78835554 +0000 UTC m=+1778.367052985" watchObservedRunningTime="2026-07-29 00:37:43.790786243 +0000 UTC m=+1778.369483688"
Jul 29 00:37:43.811677 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.808028    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6" podStartSLOduration=1.4958402259999999 podStartE2EDuration="1.808009047s" totalImagesPullingTime="312.168821ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:37:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:43.805511911 +0000 UTC m=+1778.384209359" watchObservedRunningTime="2026-07-29 00:37:43.808009047 +0000 UTC m=+1778.386706493"
Jul 29 00:37:43.829105 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:43.829060    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf" podStartSLOduration=1.8290456370000001 podStartE2EDuration="1.829045637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:37:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:37:43.827906545 +0000 UTC m=+1778.406603971" watchObservedRunningTime="2026-07-29 00:37:43.829045637 +0000 UTC m=+1778.407743081"
Jul 29 00:37:44.553187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.553103    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:44.665980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.665945    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-credential-socket\") pod \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") "
Jul 29 00:37:44.666169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666004    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-certs\") pod \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") "
Jul 29 00:37:44.666169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666037    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istiod-ca-cert\") pod \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") "
Jul 29 00:37:44.666169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666067    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-envoy\") pod \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") "
Jul 29 00:37:44.666169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666096    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-data\") pod \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") "
Jul 29 00:37:44.666169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666122    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-token\") pod \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") "
Jul 29 00:37:44.666169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666147    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-socket\") pod \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") "
Jul 29 00:37:44.666525 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666187    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vl92\" (UniqueName: \"kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-kube-api-access-2vl92\") pod \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") "
Jul 29 00:37:44.666525 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666217    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-podinfo\") pod \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\" (UID: \"d7becbe3-0fb7-4165-84d9-8c38d897d5ea\") "
Jul 29 00:37:44.666525 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666305    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7becbe3-0fb7-4165-84d9-8c38d897d5ea" (UID: "d7becbe3-0fb7-4165-84d9-8c38d897d5ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:37:44.666525 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666398    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7becbe3-0fb7-4165-84d9-8c38d897d5ea" (UID: "d7becbe3-0fb7-4165-84d9-8c38d897d5ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:37:44.666525 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666493    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7becbe3-0fb7-4165-84d9-8c38d897d5ea" (UID: "d7becbe3-0fb7-4165-84d9-8c38d897d5ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:37:44.666824 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666551    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-credential-socket\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:44.666824 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666570    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:44.666824 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.666781    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7becbe3-0fb7-4165-84d9-8c38d897d5ea" (UID: "d7becbe3-0fb7-4165-84d9-8c38d897d5ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:37:44.668354 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.668320    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7becbe3-0fb7-4165-84d9-8c38d897d5ea" (UID: "d7becbe3-0fb7-4165-84d9-8c38d897d5ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:37:44.668824 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.668775    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7becbe3-0fb7-4165-84d9-8c38d897d5ea" (UID: "d7becbe3-0fb7-4165-84d9-8c38d897d5ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 00:37:44.669172 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.669142    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-kube-api-access-2vl92" (OuterVolumeSpecName: "kube-api-access-2vl92") pod "d7becbe3-0fb7-4165-84d9-8c38d897d5ea" (UID: "d7becbe3-0fb7-4165-84d9-8c38d897d5ea"). InnerVolumeSpecName "kube-api-access-2vl92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:37:44.669352 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.669329    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7becbe3-0fb7-4165-84d9-8c38d897d5ea" (UID: "d7becbe3-0fb7-4165-84d9-8c38d897d5ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:37:44.669635 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.669615    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7becbe3-0fb7-4165-84d9-8c38d897d5ea" (UID: "d7becbe3-0fb7-4165-84d9-8c38d897d5ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:37:44.767126 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.767094    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-envoy\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:44.767126 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.767118    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-data\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:44.767126 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.767127    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-token\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:44.767126 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.767134    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-workload-socket\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:44.767371 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.767143    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vl92\" (UniqueName: \"kubernetes.io/projected/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-kube-api-access-2vl92\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:44.767371 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.767153    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istio-podinfo\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:44.767371 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.767160    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7becbe3-0fb7-4165-84d9-8c38d897d5ea-istiod-ca-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:37:44.784222 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.784189    2557 generic.go:356] "Generic (PLEG): container finished" podID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerID="6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06" exitCode=0
Jul 29 00:37:44.784346 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.784230    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"
Jul 29 00:37:44.784346 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.784263    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" event={"ID":"d7becbe3-0fb7-4165-84d9-8c38d897d5ea","Type":"ContainerDied","Data":"6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06"}
Jul 29 00:37:44.784346 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.784316    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw" event={"ID":"d7becbe3-0fb7-4165-84d9-8c38d897d5ea","Type":"ContainerDied","Data":"6eca253f9e2a85b6d05f14804216c52363f35df18cae88cec93427fd21f5a5da"}
Jul 29 00:37:44.784346 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.784338    2557 scope.go:117] "RemoveContainer" containerID="6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06"
Jul 29 00:37:44.796484 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.796463    2557 scope.go:117] "RemoveContainer" containerID="5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774"
Jul 29 00:37:44.805137 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.805112    2557 scope.go:117] "RemoveContainer" containerID="6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06"
Jul 29 00:37:44.805430 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:37:44.805412    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06\": container with ID starting with 6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06 not found: ID does not exist" containerID="6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06"
Jul 29 00:37:44.805519 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.805441    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06"} err="failed to get container status \"6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06\": rpc error: code = NotFound desc = could not find container \"6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06\": container with ID starting with 6ec1acfa1baeccfa4a0c8ea6681958ed0bed6f964eb463a884c9f98a15cd6a06 not found: ID does not exist"
Jul 29 00:37:44.805519 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.805464    2557 scope.go:117] "RemoveContainer" containerID="5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774"
Jul 29 00:37:44.805774 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:37:44.805755    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774\": container with ID starting with 5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774 not found: ID does not exist" containerID="5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774"
Jul 29 00:37:44.805827 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.805780    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774"} err="failed to get container status \"5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774\": rpc error: code = NotFound desc = could not find container \"5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774\": container with ID starting with 5f5517d68b89f714a44d5edfcfe95cc6a891d1cbef601c3042f4b16acec6e774 not found: ID does not exist"
Jul 29 00:37:44.806500 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.806481    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"]
Jul 29 00:37:44.813105 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:44.813081    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a969ca8e-openshift-default-76f9b879bd-fgrzw"]
Jul 29 00:37:45.940994 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:45.940959    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" path="/var/lib/kubelet/pods/d7becbe3-0fb7-4165-84d9-8c38d897d5ea/volumes"
Jul 29 00:37:49.790808 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:49.790772    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:37:54.786983 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:54.786950    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:37:54.787357 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:37:54.787025    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:38:05.906135 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:05.906105    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:38:05.908140 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:05.908116    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:38:12.875309 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:12.875276    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"]
Jul 29 00:38:12.875806 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:12.875530    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6" podUID="0ae3c7bf-ee97-4985-9cca-b693d61fa7bb" containerName="maas-api" containerID="cri-o://97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e" gracePeriod=30
Jul 29 00:38:12.928303 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:12.928271    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"]
Jul 29 00:38:12.928491 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:12.928468    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf" podUID="63bde123-fa4f-45fc-9132-dfe83a48fcc0" containerName="payload-processing" containerID="cri-o://50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a" gracePeriod=30
Jul 29 00:38:12.936319 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:12.936288    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"]
Jul 29 00:38:12.936544 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:12.936502    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8" podUID="324ae4dd-72ed-47ac-ac28-0c7039bffb46" containerName="payload-pre-processing" containerID="cri-o://5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a" gracePeriod=30
Jul 29 00:38:13.218794 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.218769    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:38:13.239486 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.239465    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:38:13.246114 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.246092    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:38:13.316374 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.316346    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9zb6\" (UniqueName: \"kubernetes.io/projected/324ae4dd-72ed-47ac-ac28-0c7039bffb46-kube-api-access-t9zb6\") pod \"324ae4dd-72ed-47ac-ac28-0c7039bffb46\" (UID: \"324ae4dd-72ed-47ac-ac28-0c7039bffb46\") "
Jul 29 00:38:13.316531 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.316438    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/324ae4dd-72ed-47ac-ac28-0c7039bffb46-plugins-config-volume\") pod \"324ae4dd-72ed-47ac-ac28-0c7039bffb46\" (UID: \"324ae4dd-72ed-47ac-ac28-0c7039bffb46\") "
Jul 29 00:38:13.316821 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.316797    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/324ae4dd-72ed-47ac-ac28-0c7039bffb46-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "324ae4dd-72ed-47ac-ac28-0c7039bffb46" (UID: "324ae4dd-72ed-47ac-ac28-0c7039bffb46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:38:13.318426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.318406    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/324ae4dd-72ed-47ac-ac28-0c7039bffb46-kube-api-access-t9zb6" (OuterVolumeSpecName: "kube-api-access-t9zb6") pod "324ae4dd-72ed-47ac-ac28-0c7039bffb46" (UID: "324ae4dd-72ed-47ac-ac28-0c7039bffb46"). InnerVolumeSpecName "kube-api-access-t9zb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:38:13.417324 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.417250    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63bde123-fa4f-45fc-9132-dfe83a48fcc0-plugins-config-volume\") pod \"63bde123-fa4f-45fc-9132-dfe83a48fcc0\" (UID: \"63bde123-fa4f-45fc-9132-dfe83a48fcc0\") "
Jul 29 00:38:13.417324 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.417290    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-maas-api-tls\") pod \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") "
Jul 29 00:38:13.417552 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.417333    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wglqk\" (UniqueName: \"kubernetes.io/projected/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-kube-api-access-wglqk\") pod \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") "
Jul 29 00:38:13.417552 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.417381    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54b62\" (UniqueName: \"kubernetes.io/projected/63bde123-fa4f-45fc-9132-dfe83a48fcc0-kube-api-access-54b62\") pod \"63bde123-fa4f-45fc-9132-dfe83a48fcc0\" (UID: \"63bde123-fa4f-45fc-9132-dfe83a48fcc0\") "
Jul 29 00:38:13.417552 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.417446    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-openshift-service-ca\") pod \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\" (UID: \"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb\") "
Jul 29 00:38:13.417755 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.417675    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63bde123-fa4f-45fc-9132-dfe83a48fcc0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "63bde123-fa4f-45fc-9132-dfe83a48fcc0" (UID: "63bde123-fa4f-45fc-9132-dfe83a48fcc0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:38:13.417755 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.417708    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9zb6\" (UniqueName: \"kubernetes.io/projected/324ae4dd-72ed-47ac-ac28-0c7039bffb46-kube-api-access-t9zb6\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:38:13.417755 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.417728    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/324ae4dd-72ed-47ac-ac28-0c7039bffb46-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:38:13.417881 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.417864    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ae3c7bf-ee97-4985-9cca-b693d61fa7bb" (UID: "0ae3c7bf-ee97-4985-9cca-b693d61fa7bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:38:13.419407 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.419386    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ae3c7bf-ee97-4985-9cca-b693d61fa7bb" (UID: "0ae3c7bf-ee97-4985-9cca-b693d61fa7bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:38:13.419505 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.419487    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-kube-api-access-wglqk" (OuterVolumeSpecName: "kube-api-access-wglqk") pod "0ae3c7bf-ee97-4985-9cca-b693d61fa7bb" (UID: "0ae3c7bf-ee97-4985-9cca-b693d61fa7bb"). InnerVolumeSpecName "kube-api-access-wglqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:38:13.419553 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.419511    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63bde123-fa4f-45fc-9132-dfe83a48fcc0-kube-api-access-54b62" (OuterVolumeSpecName: "kube-api-access-54b62") pod "63bde123-fa4f-45fc-9132-dfe83a48fcc0" (UID: "63bde123-fa4f-45fc-9132-dfe83a48fcc0"). InnerVolumeSpecName "kube-api-access-54b62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:38:13.518892 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.518857    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:38:13.518892 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.518887    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63bde123-fa4f-45fc-9132-dfe83a48fcc0-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:38:13.518892 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.518899    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:38:13.519119 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.518909    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wglqk\" (UniqueName: \"kubernetes.io/projected/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb-kube-api-access-wglqk\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:38:13.519119 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.518918    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54b62\" (UniqueName: \"kubernetes.io/projected/63bde123-fa4f-45fc-9132-dfe83a48fcc0-kube-api-access-54b62\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:38:13.927301 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.927267    2557 generic.go:356] "Generic (PLEG): container finished" podID="324ae4dd-72ed-47ac-ac28-0c7039bffb46" containerID="5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a" exitCode=0
Jul 29 00:38:13.927732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.927312    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"
Jul 29 00:38:13.927732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.927355    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8" event={"ID":"324ae4dd-72ed-47ac-ac28-0c7039bffb46","Type":"ContainerDied","Data":"5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a"}
Jul 29 00:38:13.927732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.927423    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8" event={"ID":"324ae4dd-72ed-47ac-ac28-0c7039bffb46","Type":"ContainerDied","Data":"c981f0cb57f9a7e4d082965efd2fea9cc17f14ae8af4ac062d81d9103cafa5e7"}
Jul 29 00:38:13.927732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.927448    2557 scope.go:117] "RemoveContainer" containerID="5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a"
Jul 29 00:38:13.928847 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.928814    2557 generic.go:356] "Generic (PLEG): container finished" podID="63bde123-fa4f-45fc-9132-dfe83a48fcc0" containerID="50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a" exitCode=0
Jul 29 00:38:13.928945 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.928854    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"
Jul 29 00:38:13.928945 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.928855    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf" event={"ID":"63bde123-fa4f-45fc-9132-dfe83a48fcc0","Type":"ContainerDied","Data":"50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a"}
Jul 29 00:38:13.928945 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.928911    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf" event={"ID":"63bde123-fa4f-45fc-9132-dfe83a48fcc0","Type":"ContainerDied","Data":"a09c0c13ba120a4ac97fae9c07e64ef43680ca2c0c223c1178561747609603b1"}
Jul 29 00:38:13.930493 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.930466    2557 generic.go:356] "Generic (PLEG): container finished" podID="0ae3c7bf-ee97-4985-9cca-b693d61fa7bb" containerID="97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e" exitCode=0
Jul 29 00:38:13.930587 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.930507    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"
Jul 29 00:38:13.930587 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.930522    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6" event={"ID":"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb","Type":"ContainerDied","Data":"97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e"}
Jul 29 00:38:13.930587 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.930563    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6" event={"ID":"0ae3c7bf-ee97-4985-9cca-b693d61fa7bb","Type":"ContainerDied","Data":"2f84fd0ef9c8512b6aa2733b79d37f4039e23d9d1efa588d93c8d7f460dec552"}
Jul 29 00:38:13.936959 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.936940    2557 scope.go:117] "RemoveContainer" containerID="5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a"
Jul 29 00:38:13.937521 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:38:13.937388    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a\": container with ID starting with 5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a not found: ID does not exist" containerID="5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a"
Jul 29 00:38:13.937521 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.937423    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a"} err="failed to get container status \"5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a\": rpc error: code = NotFound desc = could not find container \"5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a\": container with ID starting with 5ca32f299aef410c8853aa9c0012b2ab0c5442d172775f26f8b3ad495fdc4d6a not found: ID does not exist"
Jul 29 00:38:13.937521 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.937446    2557 scope.go:117] "RemoveContainer" containerID="50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a"
Jul 29 00:38:13.946792 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.946772    2557 scope.go:117] "RemoveContainer" containerID="50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a"
Jul 29 00:38:13.947057 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:38:13.947036    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a\": container with ID starting with 50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a not found: ID does not exist" containerID="50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a"
Jul 29 00:38:13.947145 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.947062    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a"} err="failed to get container status \"50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a\": rpc error: code = NotFound desc = could not find container \"50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a\": container with ID starting with 50fa218862502d3344cd201e46a242fbb8a3bf0c6360f3918362992c3f2adc5a not found: ID does not exist"
Jul 29 00:38:13.947145 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.947077    2557 scope.go:117] "RemoveContainer" containerID="97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e"
Jul 29 00:38:13.950915 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.950893    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"]
Jul 29 00:38:13.957382 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.957361    2557 scope.go:117] "RemoveContainer" containerID="97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e"
Jul 29 00:38:13.957477 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.957401    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-a0eed2d7-6489cf6948-r4mx8"]
Jul 29 00:38:13.957635 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:38:13.957617    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e\": container with ID starting with 97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e not found: ID does not exist" containerID="97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e"
Jul 29 00:38:13.957723 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.957644    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e"} err="failed to get container status \"97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e\": rpc error: code = NotFound desc = could not find container \"97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e\": container with ID starting with 97193b69a0aba8b2c0ca127d1f32968c6b97e4d87f2423769b2d70cfece2e88e not found: ID does not exist"
Jul 29 00:38:13.971808 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.971786    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"]
Jul 29 00:38:13.975923 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.975901    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a0eed2d7-5ff5fccd59-886x6"]
Jul 29 00:38:13.991440 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.991415    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"]
Jul 29 00:38:13.999505 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:13.999481    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a0eed2d7-7c8d858c74-d6kzf"]
Jul 29 00:38:15.941653 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:15.941619    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ae3c7bf-ee97-4985-9cca-b693d61fa7bb" path="/var/lib/kubelet/pods/0ae3c7bf-ee97-4985-9cca-b693d61fa7bb/volumes"
Jul 29 00:38:15.942051 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:15.942005    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="324ae4dd-72ed-47ac-ac28-0c7039bffb46" path="/var/lib/kubelet/pods/324ae4dd-72ed-47ac-ac28-0c7039bffb46/volumes"
Jul 29 00:38:15.942337 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:38:15.942323    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63bde123-fa4f-45fc-9132-dfe83a48fcc0" path="/var/lib/kubelet/pods/63bde123-fa4f-45fc-9132-dfe83a48fcc0/volumes"
Jul 29 00:40:09.434851 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.434766    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"]
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435128    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435143    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435155    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae3c7bf-ee97-4985-9cca-b693d61fa7bb" containerName="maas-api"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435163    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae3c7bf-ee97-4985-9cca-b693d61fa7bb" containerName="maas-api"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435190    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435201    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435209    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63bde123-fa4f-45fc-9132-dfe83a48fcc0" containerName="payload-processing"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435216    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="63bde123-fa4f-45fc-9132-dfe83a48fcc0" containerName="payload-processing"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435226    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="324ae4dd-72ed-47ac-ac28-0c7039bffb46" containerName="payload-pre-processing"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435234    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="324ae4dd-72ed-47ac-ac28-0c7039bffb46" containerName="payload-pre-processing"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435301    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae3c7bf-ee97-4985-9cca-b693d61fa7bb" containerName="maas-api"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435313    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="324ae4dd-72ed-47ac-ac28-0c7039bffb46" containerName="payload-pre-processing"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435321    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="63bde123-fa4f-45fc-9132-dfe83a48fcc0" containerName="payload-processing"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435328    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy"
Jul 29 00:40:09.435426 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.435334    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7becbe3-0fb7-4165-84d9-8c38d897d5ea" containerName="istio-proxy"
Jul 29 00:40:09.438324 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.438306    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:09.440209 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.440183    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bce29f49-serving-cert\""
Jul 29 00:40:09.450312 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.450289    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"]
Jul 29 00:40:09.534753 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.534722    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a692e6-6f27-4c18-aa79-a4c30599883f-openshift-service-ca\") pod \"maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:09.534942 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.534776    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a692e6-6f27-4c18-aa79-a4c30599883f-maas-api-tls\") pod \"maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:09.534942 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.534858    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hn4n\" (UniqueName: \"kubernetes.io/projected/98a692e6-6f27-4c18-aa79-a4c30599883f-kube-api-access-2hn4n\") pod \"maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:09.636073 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.636037    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a692e6-6f27-4c18-aa79-a4c30599883f-maas-api-tls\") pod \"maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:09.636257 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.636082    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hn4n\" (UniqueName: \"kubernetes.io/projected/98a692e6-6f27-4c18-aa79-a4c30599883f-kube-api-access-2hn4n\") pod \"maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:09.636257 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.636133    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a692e6-6f27-4c18-aa79-a4c30599883f-openshift-service-ca\") pod \"maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:09.636257 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:40:09.636193    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-serving-cert: secret "maas-api-e2e-mt-bce29f49-serving-cert" not found
Jul 29 00:40:09.636410 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:40:09.636271    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98a692e6-6f27-4c18-aa79-a4c30599883f-maas-api-tls podName:98a692e6-6f27-4c18-aa79-a4c30599883f nodeName:}" failed. No retries permitted until 2026-07-29 00:40:10.136250206 +0000 UTC m=+1924.714947645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/98a692e6-6f27-4c18-aa79-a4c30599883f-maas-api-tls") pod "maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww" (UID: "98a692e6-6f27-4c18-aa79-a4c30599883f") : secret "maas-api-e2e-mt-bce29f49-serving-cert" not found
Jul 29 00:40:09.636790 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.636773    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a692e6-6f27-4c18-aa79-a4c30599883f-openshift-service-ca\") pod \"maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:09.662266 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:09.662241    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hn4n\" (UniqueName: \"kubernetes.io/projected/98a692e6-6f27-4c18-aa79-a4c30599883f-kube-api-access-2hn4n\") pod \"maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:10.140056 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.140026    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a692e6-6f27-4c18-aa79-a4c30599883f-maas-api-tls\") pod \"maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:10.142381 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.142356    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a692e6-6f27-4c18-aa79-a4c30599883f-maas-api-tls\") pod \"maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:10.146731 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.146709    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"]
Jul 29 00:40:10.150354 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.150333    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:10.151974 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.151957    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-bce29f49\""
Jul 29 00:40:10.152071 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.152019    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-bce29f49-dockercfg-p28lm\""
Jul 29 00:40:10.160324 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.160304    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"]
Jul 29 00:40:10.197006 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.196983    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"]
Jul 29 00:40:10.200099 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.200084    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:10.209141 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.209120    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"]
Jul 29 00:40:10.240636 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.240612    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6\" (UID: \"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:10.240773 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.240739    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfhzf\" (UniqueName: \"kubernetes.io/projected/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-kube-api-access-zfhzf\") pod \"payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6\" (UID: \"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:10.341991 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.341959    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6\" (UID: \"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:10.342173 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.342050    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfhzf\" (UniqueName: \"kubernetes.io/projected/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-kube-api-access-zfhzf\") pod \"payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6\" (UID: \"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:10.342173 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.342078    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-plugins-config-volume\") pod \"payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx\" (UID: \"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e\") " pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:10.342173 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.342126    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7k5s\" (UniqueName: \"kubernetes.io/projected/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-kube-api-access-w7k5s\") pod \"payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx\" (UID: \"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e\") " pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:10.342743 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.342713    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6\" (UID: \"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:10.349135 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.349108    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfhzf\" (UniqueName: \"kubernetes.io/projected/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-kube-api-access-zfhzf\") pod \"payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6\" (UID: \"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:10.351283 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.351257    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:10.442773 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.442742    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-plugins-config-volume\") pod \"payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx\" (UID: \"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e\") " pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:10.443181 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.442798    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7k5s\" (UniqueName: \"kubernetes.io/projected/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-kube-api-access-w7k5s\") pod \"payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx\" (UID: \"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e\") " pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:10.443873 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.443848    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-plugins-config-volume\") pod \"payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx\" (UID: \"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e\") " pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:10.449732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.449704    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7k5s\" (UniqueName: \"kubernetes.io/projected/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-kube-api-access-w7k5s\") pod \"payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx\" (UID: \"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e\") " pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:10.461753 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.461617    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:10.486435 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.486412    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"]
Jul 29 00:40:10.488737 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:40:10.488709    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a692e6_6f27_4c18_aa79_a4c30599883f.slice/crio-663d29c262566f5e874f7108a1dc3205d05ca5d6630e5b79a95914a455a1f03c WatchSource:0}: Error finding container 663d29c262566f5e874f7108a1dc3205d05ca5d6630e5b79a95914a455a1f03c: Status 404 returned error can't find the container with id 663d29c262566f5e874f7108a1dc3205d05ca5d6630e5b79a95914a455a1f03c
Jul 29 00:40:10.509760 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.509735    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:10.605211 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.605188    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"]
Jul 29 00:40:10.609218 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:40:10.609178    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf53f9dcc_1fe4_49e1_b0fa_29d7b124259d.slice/crio-a9965df0f89ace92498d0c06e3b7146e83b7d78e0861601fa0aabba551dc0a15 WatchSource:0}: Error finding container a9965df0f89ace92498d0c06e3b7146e83b7d78e0861601fa0aabba551dc0a15: Status 404 returned error can't find the container with id a9965df0f89ace92498d0c06e3b7146e83b7d78e0861601fa0aabba551dc0a15
Jul 29 00:40:10.662183 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:10.662091    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"]
Jul 29 00:40:10.664491 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:40:10.664467    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6bd70c_504c_4ccd_a79d_2e994ef8ac2e.slice/crio-a4aebc0cb0812dcb14a1166f2602e87436a97e2fbb468ee7cf6bb382bbe1b4a1 WatchSource:0}: Error finding container a4aebc0cb0812dcb14a1166f2602e87436a97e2fbb468ee7cf6bb382bbe1b4a1: Status 404 returned error can't find the container with id a4aebc0cb0812dcb14a1166f2602e87436a97e2fbb468ee7cf6bb382bbe1b4a1
Jul 29 00:40:11.473912 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.473844    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww" event={"ID":"98a692e6-6f27-4c18-aa79-a4c30599883f","Type":"ContainerStarted","Data":"f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1"}
Jul 29 00:40:11.474367 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.473946    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww" event={"ID":"98a692e6-6f27-4c18-aa79-a4c30599883f","Type":"ContainerStarted","Data":"663d29c262566f5e874f7108a1dc3205d05ca5d6630e5b79a95914a455a1f03c"}
Jul 29 00:40:11.474367 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.474082    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:11.475809 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.475779    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6" event={"ID":"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d","Type":"ContainerStarted","Data":"de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71"}
Jul 29 00:40:11.475985 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.475908    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:11.475985 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.475934    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6" event={"ID":"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d","Type":"ContainerStarted","Data":"a9965df0f89ace92498d0c06e3b7146e83b7d78e0861601fa0aabba551dc0a15"}
Jul 29 00:40:11.477421 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.477395    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx" event={"ID":"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e","Type":"ContainerStarted","Data":"2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8"}
Jul 29 00:40:11.477537 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.477438    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx" event={"ID":"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e","Type":"ContainerStarted","Data":"a4aebc0cb0812dcb14a1166f2602e87436a97e2fbb468ee7cf6bb382bbe1b4a1"}
Jul 29 00:40:11.477537 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.477483    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:11.494599 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.494559    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww" podStartSLOduration=1.933699518 podStartE2EDuration="2.494546674s" totalImagesPullingTime="560.847156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:40:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:40:11.49329841 +0000 UTC m=+1926.071995853" watchObservedRunningTime="2026-07-29 00:40:11.494546674 +0000 UTC m=+1926.073244117"
Jul 29 00:40:11.507623 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.507582    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6" podStartSLOduration=1.5075700950000002 podStartE2EDuration="1.507570095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:40:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:40:11.505762465 +0000 UTC m=+1926.084459913" watchObservedRunningTime="2026-07-29 00:40:11.507570095 +0000 UTC m=+1926.086267540"
Jul 29 00:40:11.518594 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:11.518553    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx" podStartSLOduration=1.5185415949999999 podStartE2EDuration="1.518541595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:40:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:40:11.517940716 +0000 UTC m=+1926.096638160" watchObservedRunningTime="2026-07-29 00:40:11.518541595 +0000 UTC m=+1926.097239041"
Jul 29 00:40:17.487689 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:17.487642    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:22.484530 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:22.484501    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:22.484917 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:22.484566    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:51.840349 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:51.839783    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"]
Jul 29 00:40:51.840349 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:51.840033    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww" podUID="98a692e6-6f27-4c18-aa79-a4c30599883f" containerName="maas-api" containerID="cri-o://f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1" gracePeriod=30
Jul 29 00:40:51.896969 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:51.896935    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"]
Jul 29 00:40:51.897220 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:51.897189    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx" podUID="4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e" containerName="payload-processing" containerID="cri-o://2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8" gracePeriod=30
Jul 29 00:40:51.907179 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:51.907147    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"]
Jul 29 00:40:51.907555 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:51.907526    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6" podUID="f53f9dcc-1fe4-49e1-b0fa-29d7b124259d" containerName="payload-pre-processing" containerID="cri-o://de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71" gracePeriod=30
Jul 29 00:40:52.167623 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.167593    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:52.184939 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.184909    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:52.206617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.206589    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a692e6-6f27-4c18-aa79-a4c30599883f-openshift-service-ca\") pod \"98a692e6-6f27-4c18-aa79-a4c30599883f\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") "
Jul 29 00:40:52.206771 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.206648    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a692e6-6f27-4c18-aa79-a4c30599883f-maas-api-tls\") pod \"98a692e6-6f27-4c18-aa79-a4c30599883f\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") "
Jul 29 00:40:52.206771 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.206718    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hn4n\" (UniqueName: \"kubernetes.io/projected/98a692e6-6f27-4c18-aa79-a4c30599883f-kube-api-access-2hn4n\") pod \"98a692e6-6f27-4c18-aa79-a4c30599883f\" (UID: \"98a692e6-6f27-4c18-aa79-a4c30599883f\") "
Jul 29 00:40:52.207003 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.206962    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a692e6-6f27-4c18-aa79-a4c30599883f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98a692e6-6f27-4c18-aa79-a4c30599883f" (UID: "98a692e6-6f27-4c18-aa79-a4c30599883f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:40:52.207120 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.207036    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98a692e6-6f27-4c18-aa79-a4c30599883f-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:40:52.209264 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.209242    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98a692e6-6f27-4c18-aa79-a4c30599883f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98a692e6-6f27-4c18-aa79-a4c30599883f" (UID: "98a692e6-6f27-4c18-aa79-a4c30599883f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:40:52.209354 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.209271    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a692e6-6f27-4c18-aa79-a4c30599883f-kube-api-access-2hn4n" (OuterVolumeSpecName: "kube-api-access-2hn4n") pod "98a692e6-6f27-4c18-aa79-a4c30599883f" (UID: "98a692e6-6f27-4c18-aa79-a4c30599883f"). InnerVolumeSpecName "kube-api-access-2hn4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:40:52.213474 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.213453    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:52.308411 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.308377    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-plugins-config-volume\") pod \"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e\" (UID: \"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e\") "
Jul 29 00:40:52.308581 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.308427    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7k5s\" (UniqueName: \"kubernetes.io/projected/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-kube-api-access-w7k5s\") pod \"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e\" (UID: \"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e\") "
Jul 29 00:40:52.308581 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.308453    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfhzf\" (UniqueName: \"kubernetes.io/projected/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-kube-api-access-zfhzf\") pod \"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d\" (UID: \"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d\") "
Jul 29 00:40:52.308581 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.308475    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-plugins-config-volume\") pod \"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d\" (UID: \"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d\") "
Jul 29 00:40:52.308732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.308620    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98a692e6-6f27-4c18-aa79-a4c30599883f-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:40:52.308732 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.308637    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hn4n\" (UniqueName: \"kubernetes.io/projected/98a692e6-6f27-4c18-aa79-a4c30599883f-kube-api-access-2hn4n\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:40:52.308847 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.308818    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e" (UID: "4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:40:52.308925 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.308906    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f53f9dcc-1fe4-49e1-b0fa-29d7b124259d" (UID: "f53f9dcc-1fe4-49e1-b0fa-29d7b124259d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:40:52.310650 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.310621    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-kube-api-access-w7k5s" (OuterVolumeSpecName: "kube-api-access-w7k5s") pod "4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e" (UID: "4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e"). InnerVolumeSpecName "kube-api-access-w7k5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:40:52.310650 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.310690    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-kube-api-access-zfhzf" (OuterVolumeSpecName: "kube-api-access-zfhzf") pod "f53f9dcc-1fe4-49e1-b0fa-29d7b124259d" (UID: "f53f9dcc-1fe4-49e1-b0fa-29d7b124259d"). InnerVolumeSpecName "kube-api-access-zfhzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:40:52.409330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.409245    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7k5s\" (UniqueName: \"kubernetes.io/projected/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-kube-api-access-w7k5s\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:40:52.409330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.409276    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfhzf\" (UniqueName: \"kubernetes.io/projected/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-kube-api-access-zfhzf\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:40:52.409330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.409285    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:40:52.409330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.409293    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:40:52.674763 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.674678    2557 generic.go:356] "Generic (PLEG): container finished" podID="f53f9dcc-1fe4-49e1-b0fa-29d7b124259d" containerID="de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71" exitCode=0
Jul 29 00:40:52.674763 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.674731    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"
Jul 29 00:40:52.674986 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.674761    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6" event={"ID":"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d","Type":"ContainerDied","Data":"de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71"}
Jul 29 00:40:52.674986 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.674842    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6" event={"ID":"f53f9dcc-1fe4-49e1-b0fa-29d7b124259d","Type":"ContainerDied","Data":"a9965df0f89ace92498d0c06e3b7146e83b7d78e0861601fa0aabba551dc0a15"}
Jul 29 00:40:52.674986 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.674869    2557 scope.go:117] "RemoveContainer" containerID="de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71"
Jul 29 00:40:52.676097 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.676069    2557 generic.go:356] "Generic (PLEG): container finished" podID="4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e" containerID="2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8" exitCode=0
Jul 29 00:40:52.676233 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.676112    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"
Jul 29 00:40:52.676233 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.676153    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx" event={"ID":"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e","Type":"ContainerDied","Data":"2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8"}
Jul 29 00:40:52.676233 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.676200    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx" event={"ID":"4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e","Type":"ContainerDied","Data":"a4aebc0cb0812dcb14a1166f2602e87436a97e2fbb468ee7cf6bb382bbe1b4a1"}
Jul 29 00:40:52.677698 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.677671    2557 generic.go:356] "Generic (PLEG): container finished" podID="98a692e6-6f27-4c18-aa79-a4c30599883f" containerID="f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1" exitCode=0
Jul 29 00:40:52.677815 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.677729    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww" event={"ID":"98a692e6-6f27-4c18-aa79-a4c30599883f","Type":"ContainerDied","Data":"f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1"}
Jul 29 00:40:52.677815 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.677769    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww" event={"ID":"98a692e6-6f27-4c18-aa79-a4c30599883f","Type":"ContainerDied","Data":"663d29c262566f5e874f7108a1dc3205d05ca5d6630e5b79a95914a455a1f03c"}
Jul 29 00:40:52.677815 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.677778    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"
Jul 29 00:40:52.685127 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.684937    2557 scope.go:117] "RemoveContainer" containerID="de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71"
Jul 29 00:40:52.685297 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:40:52.685268    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71\": container with ID starting with de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71 not found: ID does not exist" containerID="de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71"
Jul 29 00:40:52.685387 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.685305    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71"} err="failed to get container status \"de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71\": rpc error: code = NotFound desc = could not find container \"de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71\": container with ID starting with de97bb22a8213e39949a7c477c0f1eba58caf00d8480d362bcbd2a3b86370c71 not found: ID does not exist"
Jul 29 00:40:52.685387 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.685329    2557 scope.go:117] "RemoveContainer" containerID="2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8"
Jul 29 00:40:52.693865 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.693850    2557 scope.go:117] "RemoveContainer" containerID="2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8"
Jul 29 00:40:52.694110 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:40:52.694091    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8\": container with ID starting with 2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8 not found: ID does not exist" containerID="2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8"
Jul 29 00:40:52.694153 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.694116    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8"} err="failed to get container status \"2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8\": rpc error: code = NotFound desc = could not find container \"2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8\": container with ID starting with 2ef6962f32e8772c82122e2da1d9fa2e0ff0c455381210a5aa778d33f30590b8 not found: ID does not exist"
Jul 29 00:40:52.694153 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.694132    2557 scope.go:117] "RemoveContainer" containerID="f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1"
Jul 29 00:40:52.699823 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.699802    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"]
Jul 29 00:40:52.703764 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.703744    2557 scope.go:117] "RemoveContainer" containerID="f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1"
Jul 29 00:40:52.704025 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:40:52.704007    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1\": container with ID starting with f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1 not found: ID does not exist" containerID="f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1"
Jul 29 00:40:52.704093 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.704035    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1"} err="failed to get container status \"f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1\": rpc error: code = NotFound desc = could not find container \"f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1\": container with ID starting with f7664cdc068a5938a54bea4c309292fcf142498c79072168962ba1a481202bf1 not found: ID does not exist"
Jul 29 00:40:52.704093 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.704067    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bce29f49-96cd67fd4-qx2j6"]
Jul 29 00:40:52.712874 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.712847    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"]
Jul 29 00:40:52.714924 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.714904    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bce29f49-76b6455c74-mqdsx"]
Jul 29 00:40:52.723158 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.723126    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"]
Jul 29 00:40:52.729842 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:52.729823    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bce29f49-595bf779b9-zc2ww"]
Jul 29 00:40:53.940789 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:53.940755    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e" path="/var/lib/kubelet/pods/4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e/volumes"
Jul 29 00:40:53.941158 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:53.941146    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98a692e6-6f27-4c18-aa79-a4c30599883f" path="/var/lib/kubelet/pods/98a692e6-6f27-4c18-aa79-a4c30599883f/volumes"
Jul 29 00:40:53.941462 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:40:53.941449    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f53f9dcc-1fe4-49e1-b0fa-29d7b124259d" path="/var/lib/kubelet/pods/f53f9dcc-1fe4-49e1-b0fa-29d7b124259d/volumes"
Jul 29 00:41:20.817983 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.817948    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"]
Jul 29 00:41:20.818483 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.818461    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53f9dcc-1fe4-49e1-b0fa-29d7b124259d" containerName="payload-pre-processing"
Jul 29 00:41:20.818483 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.818479    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53f9dcc-1fe4-49e1-b0fa-29d7b124259d" containerName="payload-pre-processing"
Jul 29 00:41:20.818598 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.818491    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e" containerName="payload-processing"
Jul 29 00:41:20.818598 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.818499    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e" containerName="payload-processing"
Jul 29 00:41:20.818598 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.818523    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a692e6-6f27-4c18-aa79-a4c30599883f" containerName="maas-api"
Jul 29 00:41:20.818598 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.818533    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a692e6-6f27-4c18-aa79-a4c30599883f" containerName="maas-api"
Jul 29 00:41:20.818822 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.818642    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b6bd70c-504c-4ccd-a79d-2e994ef8ac2e" containerName="payload-processing"
Jul 29 00:41:20.818822 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.818673    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="98a692e6-6f27-4c18-aa79-a4c30599883f" containerName="maas-api"
Jul 29 00:41:20.818822 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.818685    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f53f9dcc-1fe4-49e1-b0fa-29d7b124259d" containerName="payload-pre-processing"
Jul 29 00:41:20.821754 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.821732    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:20.823668 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.823637    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9d8ec5a2-serving-cert\""
Jul 29 00:41:20.830604 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.830530    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"]
Jul 29 00:41:20.963402 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.963369    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ad6907-fdf7-46fc-bee8-25ddf92364c4-maas-api-tls\") pod \"maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:20.963611 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.963422    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9jfq\" (UniqueName: \"kubernetes.io/projected/06ad6907-fdf7-46fc-bee8-25ddf92364c4-kube-api-access-n9jfq\") pod \"maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:20.963611 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:20.963485    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ad6907-fdf7-46fc-bee8-25ddf92364c4-openshift-service-ca\") pod \"maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:21.064600 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.064565    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ad6907-fdf7-46fc-bee8-25ddf92364c4-maas-api-tls\") pod \"maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:21.064816 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.064608    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9jfq\" (UniqueName: \"kubernetes.io/projected/06ad6907-fdf7-46fc-bee8-25ddf92364c4-kube-api-access-n9jfq\") pod \"maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:21.064816 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.064646    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ad6907-fdf7-46fc-bee8-25ddf92364c4-openshift-service-ca\") pod \"maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:21.064816 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:41:21.064733    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-serving-cert: secret "maas-api-e2e-mt-9d8ec5a2-serving-cert" not found
Jul 29 00:41:21.064816 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:41:21.064813    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06ad6907-fdf7-46fc-bee8-25ddf92364c4-maas-api-tls podName:06ad6907-fdf7-46fc-bee8-25ddf92364c4 nodeName:}" failed. No retries permitted until 2026-07-29 00:41:21.564791499 +0000 UTC m=+1996.143488923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/06ad6907-fdf7-46fc-bee8-25ddf92364c4-maas-api-tls") pod "maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp" (UID: "06ad6907-fdf7-46fc-bee8-25ddf92364c4") : secret "maas-api-e2e-mt-9d8ec5a2-serving-cert" not found
Jul 29 00:41:21.065350 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.065330    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ad6907-fdf7-46fc-bee8-25ddf92364c4-openshift-service-ca\") pod \"maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:21.072282 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.072223    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9jfq\" (UniqueName: \"kubernetes.io/projected/06ad6907-fdf7-46fc-bee8-25ddf92364c4-kube-api-access-n9jfq\") pod \"maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:21.570097 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.570056    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ad6907-fdf7-46fc-bee8-25ddf92364c4-maas-api-tls\") pod \"maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:21.572599 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.572571    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ad6907-fdf7-46fc-bee8-25ddf92364c4-maas-api-tls\") pod \"maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:21.728870 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.728838    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"]
Jul 29 00:41:21.732427 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.732404    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:41:21.734487 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.734457    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9d8ec5a2\""
Jul 29 00:41:21.734639 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.734457    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9d8ec5a2-dockercfg-bt9br\""
Jul 29 00:41:21.734760 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.734696    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:21.741400 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.741376    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"]
Jul 29 00:41:21.761877 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.761849    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"]
Jul 29 00:41:21.765536 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.765515    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:41:21.775679 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.775643    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"]
Jul 29 00:41:21.873075 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.873043    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42826e41-2abe-45d1-82e4-1eaaff02fce3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z\" (UID: \"42826e41-2abe-45d1-82e4-1eaaff02fce3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:41:21.873075 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.873077    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-plugins-config-volume\") pod \"payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc\" (UID: \"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2\") " pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:41:21.873550 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.873114    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rr4x\" (UniqueName: \"kubernetes.io/projected/42826e41-2abe-45d1-82e4-1eaaff02fce3-kube-api-access-8rr4x\") pod \"payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z\" (UID: \"42826e41-2abe-45d1-82e4-1eaaff02fce3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:41:21.873550 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.873243    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn6ft\" (UniqueName: \"kubernetes.io/projected/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-kube-api-access-pn6ft\") pod \"payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc\" (UID: \"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2\") " pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:41:21.884415 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.884304    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"]
Jul 29 00:41:21.888278 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:41:21.888250    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ad6907_fdf7_46fc_bee8_25ddf92364c4.slice/crio-f634418ed82313969f1cc042589618fce059304fec671cd354d2eb9f99b70bf5 WatchSource:0}: Error finding container f634418ed82313969f1cc042589618fce059304fec671cd354d2eb9f99b70bf5: Status 404 returned error can't find the container with id f634418ed82313969f1cc042589618fce059304fec671cd354d2eb9f99b70bf5
Jul 29 00:41:21.889453 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.889434    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:41:21.974406 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.974372    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42826e41-2abe-45d1-82e4-1eaaff02fce3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z\" (UID: \"42826e41-2abe-45d1-82e4-1eaaff02fce3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:41:21.974406 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.974406    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-plugins-config-volume\") pod \"payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc\" (UID: \"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2\") " pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:41:21.974673 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.974445    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rr4x\" (UniqueName: \"kubernetes.io/projected/42826e41-2abe-45d1-82e4-1eaaff02fce3-kube-api-access-8rr4x\") pod \"payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z\" (UID: \"42826e41-2abe-45d1-82e4-1eaaff02fce3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:41:21.974673 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.974475    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn6ft\" (UniqueName: \"kubernetes.io/projected/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-kube-api-access-pn6ft\") pod \"payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc\" (UID: \"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2\") " pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:41:21.975214 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.975182    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42826e41-2abe-45d1-82e4-1eaaff02fce3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z\" (UID: \"42826e41-2abe-45d1-82e4-1eaaff02fce3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:41:21.975316 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.975233    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-plugins-config-volume\") pod \"payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc\" (UID: \"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2\") " pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:41:21.982162 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.982133    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rr4x\" (UniqueName: \"kubernetes.io/projected/42826e41-2abe-45d1-82e4-1eaaff02fce3-kube-api-access-8rr4x\") pod \"payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z\" (UID: \"42826e41-2abe-45d1-82e4-1eaaff02fce3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:41:21.982265 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:21.982167    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn6ft\" (UniqueName: \"kubernetes.io/projected/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-kube-api-access-pn6ft\") pod \"payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc\" (UID: \"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2\") " pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:41:22.048025 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.047995    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:41:22.079715 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.079616    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:41:22.198638 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.198603    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"]
Jul 29 00:41:22.200514 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:41:22.200446    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42826e41_2abe_45d1_82e4_1eaaff02fce3.slice/crio-5c49e5c6f47b8fbdab8bb137c300353ae74b9b1d624243b1785ea32bde3dc8f5 WatchSource:0}: Error finding container 5c49e5c6f47b8fbdab8bb137c300353ae74b9b1d624243b1785ea32bde3dc8f5: Status 404 returned error can't find the container with id 5c49e5c6f47b8fbdab8bb137c300353ae74b9b1d624243b1785ea32bde3dc8f5
Jul 29 00:41:22.228647 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.228625    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"]
Jul 29 00:41:22.236673 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:41:22.236632    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2c92b5_8c84_4254_9d4d_d4e6ba3f7dd2.slice/crio-0aa079fb8b179290abcbc091e7eae16326870cccb0c28d4751662aca1561f009 WatchSource:0}: Error finding container 0aa079fb8b179290abcbc091e7eae16326870cccb0c28d4751662aca1561f009: Status 404 returned error can't find the container with id 0aa079fb8b179290abcbc091e7eae16326870cccb0c28d4751662aca1561f009
Jul 29 00:41:22.829240 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.829193    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc" event={"ID":"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2","Type":"ContainerStarted","Data":"049298973d9815c1808e290f1be3eb18c1000607f7bfa8d3982b75500bc5c49d"}
Jul 29 00:41:22.829538 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.829263    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc" event={"ID":"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2","Type":"ContainerStarted","Data":"0aa079fb8b179290abcbc091e7eae16326870cccb0c28d4751662aca1561f009"}
Jul 29 00:41:22.829538 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.829290    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:41:22.830701 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.830645    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z" event={"ID":"42826e41-2abe-45d1-82e4-1eaaff02fce3","Type":"ContainerStarted","Data":"cde7c6bd5043fa01677ccbaaa7770db9b96ea05d630a25580b1b5fb7945b0ad0"}
Jul 29 00:41:22.830809 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.830720    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z" event={"ID":"42826e41-2abe-45d1-82e4-1eaaff02fce3","Type":"ContainerStarted","Data":"5c49e5c6f47b8fbdab8bb137c300353ae74b9b1d624243b1785ea32bde3dc8f5"}
Jul 29 00:41:22.830809 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.830754    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:41:22.832052 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.832028    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp" event={"ID":"06ad6907-fdf7-46fc-bee8-25ddf92364c4","Type":"ContainerStarted","Data":"2751117aad94284797367cf0132b2c5cf76f80c02a6c8552d48c96dcc9667ea1"}
Jul 29 00:41:22.832187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.832072    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp" event={"ID":"06ad6907-fdf7-46fc-bee8-25ddf92364c4","Type":"ContainerStarted","Data":"f634418ed82313969f1cc042589618fce059304fec671cd354d2eb9f99b70bf5"}
Jul 29 00:41:22.832187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.832111    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:22.843714 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.843461    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc" podStartSLOduration=1.843445392 podStartE2EDuration="1.843445392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:41:22.84300132 +0000 UTC m=+1997.421698759" watchObservedRunningTime="2026-07-29 00:41:22.843445392 +0000 UTC m=+1997.422142838"
Jul 29 00:41:22.856993 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.856952    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z" podStartSLOduration=1.856938711 podStartE2EDuration="1.856938711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:41:22.855483083 +0000 UTC m=+1997.434180528" watchObservedRunningTime="2026-07-29 00:41:22.856938711 +0000 UTC m=+1997.435636156"
Jul 29 00:41:22.872679 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:22.872612    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp" podStartSLOduration=2.496611811 podStartE2EDuration="2.872596724s" totalImagesPullingTime="375.984913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:41:22.870888327 +0000 UTC m=+1997.449585772" watchObservedRunningTime="2026-07-29 00:41:22.872596724 +0000 UTC m=+1997.451294173"
Jul 29 00:41:28.842372 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:28.842341    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:41:33.838972 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:33.838944    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:41:33.839381 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:41:33.839023    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:42:24.808447 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:24.808411    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"]
Jul 29 00:42:24.811724 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:24.808673    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp" podUID="06ad6907-fdf7-46fc-bee8-25ddf92364c4" containerName="maas-api" containerID="cri-o://2751117aad94284797367cf0132b2c5cf76f80c02a6c8552d48c96dcc9667ea1" gracePeriod=30
Jul 29 00:42:24.855785 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:24.855758    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"]
Jul 29 00:42:24.856031 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:24.856001    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc" podUID="fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2" containerName="payload-processing" containerID="cri-o://049298973d9815c1808e290f1be3eb18c1000607f7bfa8d3982b75500bc5c49d" gracePeriod=30
Jul 29 00:42:24.870216 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:24.870188    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"]
Jul 29 00:42:24.870450 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:24.870418    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z" podUID="42826e41-2abe-45d1-82e4-1eaaff02fce3" containerName="payload-pre-processing" containerID="cri-o://cde7c6bd5043fa01677ccbaaa7770db9b96ea05d630a25580b1b5fb7945b0ad0" gracePeriod=30
Jul 29 00:42:25.115284 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.115252    2557 generic.go:356] "Generic (PLEG): container finished" podID="42826e41-2abe-45d1-82e4-1eaaff02fce3" containerID="cde7c6bd5043fa01677ccbaaa7770db9b96ea05d630a25580b1b5fb7945b0ad0" exitCode=0
Jul 29 00:42:25.115408 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.115363    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z" event={"ID":"42826e41-2abe-45d1-82e4-1eaaff02fce3","Type":"ContainerDied","Data":"cde7c6bd5043fa01677ccbaaa7770db9b96ea05d630a25580b1b5fb7945b0ad0"}
Jul 29 00:42:25.116946 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.116919    2557 generic.go:356] "Generic (PLEG): container finished" podID="06ad6907-fdf7-46fc-bee8-25ddf92364c4" containerID="2751117aad94284797367cf0132b2c5cf76f80c02a6c8552d48c96dcc9667ea1" exitCode=0
Jul 29 00:42:25.117075 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.116999    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp" event={"ID":"06ad6907-fdf7-46fc-bee8-25ddf92364c4","Type":"ContainerDied","Data":"2751117aad94284797367cf0132b2c5cf76f80c02a6c8552d48c96dcc9667ea1"}
Jul 29 00:42:25.117141 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.117071    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp" event={"ID":"06ad6907-fdf7-46fc-bee8-25ddf92364c4","Type":"ContainerDied","Data":"f634418ed82313969f1cc042589618fce059304fec671cd354d2eb9f99b70bf5"}
Jul 29 00:42:25.117141 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.117103    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f634418ed82313969f1cc042589618fce059304fec671cd354d2eb9f99b70bf5"
Jul 29 00:42:25.118727 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.118707    2557 generic.go:356] "Generic (PLEG): container finished" podID="fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2" containerID="049298973d9815c1808e290f1be3eb18c1000607f7bfa8d3982b75500bc5c49d" exitCode=0
Jul 29 00:42:25.118837 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.118773    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc" event={"ID":"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2","Type":"ContainerDied","Data":"049298973d9815c1808e290f1be3eb18c1000607f7bfa8d3982b75500bc5c49d"}
Jul 29 00:42:25.130788 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.130765    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:42:25.163610 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.163586    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:42:25.168878 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.168859    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:42:25.239362 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.239329    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ad6907-fdf7-46fc-bee8-25ddf92364c4-maas-api-tls\") pod \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") "
Jul 29 00:42:25.239543 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.239387    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ad6907-fdf7-46fc-bee8-25ddf92364c4-openshift-service-ca\") pod \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") "
Jul 29 00:42:25.239543 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.239443    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-plugins-config-volume\") pod \"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2\" (UID: \"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2\") "
Jul 29 00:42:25.239543 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.239465    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn6ft\" (UniqueName: \"kubernetes.io/projected/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-kube-api-access-pn6ft\") pod \"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2\" (UID: \"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2\") "
Jul 29 00:42:25.239543 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.239520    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rr4x\" (UniqueName: \"kubernetes.io/projected/42826e41-2abe-45d1-82e4-1eaaff02fce3-kube-api-access-8rr4x\") pod \"42826e41-2abe-45d1-82e4-1eaaff02fce3\" (UID: \"42826e41-2abe-45d1-82e4-1eaaff02fce3\") "
Jul 29 00:42:25.239794 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.239582    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9jfq\" (UniqueName: \"kubernetes.io/projected/06ad6907-fdf7-46fc-bee8-25ddf92364c4-kube-api-access-n9jfq\") pod \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\" (UID: \"06ad6907-fdf7-46fc-bee8-25ddf92364c4\") "
Jul 29 00:42:25.239794 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.239611    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42826e41-2abe-45d1-82e4-1eaaff02fce3-plugins-config-volume\") pod \"42826e41-2abe-45d1-82e4-1eaaff02fce3\" (UID: \"42826e41-2abe-45d1-82e4-1eaaff02fce3\") "
Jul 29 00:42:25.239891 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.239834    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06ad6907-fdf7-46fc-bee8-25ddf92364c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "06ad6907-fdf7-46fc-bee8-25ddf92364c4" (UID: "06ad6907-fdf7-46fc-bee8-25ddf92364c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:42:25.239891 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.239845    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2" (UID: "fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:42:25.240061 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.240039    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ad6907-fdf7-46fc-bee8-25ddf92364c4-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:42:25.240131 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.240067    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:42:25.240131 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.240110    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42826e41-2abe-45d1-82e4-1eaaff02fce3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "42826e41-2abe-45d1-82e4-1eaaff02fce3" (UID: "42826e41-2abe-45d1-82e4-1eaaff02fce3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:42:25.241710 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.241678    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42826e41-2abe-45d1-82e4-1eaaff02fce3-kube-api-access-8rr4x" (OuterVolumeSpecName: "kube-api-access-8rr4x") pod "42826e41-2abe-45d1-82e4-1eaaff02fce3" (UID: "42826e41-2abe-45d1-82e4-1eaaff02fce3"). InnerVolumeSpecName "kube-api-access-8rr4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:42:25.241832 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.241725    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06ad6907-fdf7-46fc-bee8-25ddf92364c4-kube-api-access-n9jfq" (OuterVolumeSpecName: "kube-api-access-n9jfq") pod "06ad6907-fdf7-46fc-bee8-25ddf92364c4" (UID: "06ad6907-fdf7-46fc-bee8-25ddf92364c4"). InnerVolumeSpecName "kube-api-access-n9jfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:42:25.241832 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.241734    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-kube-api-access-pn6ft" (OuterVolumeSpecName: "kube-api-access-pn6ft") pod "fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2" (UID: "fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2"). InnerVolumeSpecName "kube-api-access-pn6ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:42:25.241832 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.241734    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06ad6907-fdf7-46fc-bee8-25ddf92364c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "06ad6907-fdf7-46fc-bee8-25ddf92364c4" (UID: "06ad6907-fdf7-46fc-bee8-25ddf92364c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:42:25.341188 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.341124    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rr4x\" (UniqueName: \"kubernetes.io/projected/42826e41-2abe-45d1-82e4-1eaaff02fce3-kube-api-access-8rr4x\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:42:25.341188 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.341145    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9jfq\" (UniqueName: \"kubernetes.io/projected/06ad6907-fdf7-46fc-bee8-25ddf92364c4-kube-api-access-n9jfq\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:42:25.341188 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.341154    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42826e41-2abe-45d1-82e4-1eaaff02fce3-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:42:25.341188 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.341162    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ad6907-fdf7-46fc-bee8-25ddf92364c4-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:42:25.341188 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:25.341173    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn6ft\" (UniqueName: \"kubernetes.io/projected/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2-kube-api-access-pn6ft\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:42:26.124457 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.124427    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"
Jul 29 00:42:26.124926 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.124454    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z" event={"ID":"42826e41-2abe-45d1-82e4-1eaaff02fce3","Type":"ContainerDied","Data":"5c49e5c6f47b8fbdab8bb137c300353ae74b9b1d624243b1785ea32bde3dc8f5"}
Jul 29 00:42:26.124926 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.124511    2557 scope.go:117] "RemoveContainer" containerID="cde7c6bd5043fa01677ccbaaa7770db9b96ea05d630a25580b1b5fb7945b0ad0"
Jul 29 00:42:26.125888 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.125802    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"
Jul 29 00:42:26.125888 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.125846    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc" event={"ID":"fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2","Type":"ContainerDied","Data":"0aa079fb8b179290abcbc091e7eae16326870cccb0c28d4751662aca1561f009"}
Jul 29 00:42:26.126021 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.126008    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"
Jul 29 00:42:26.133543 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.133525    2557 scope.go:117] "RemoveContainer" containerID="049298973d9815c1808e290f1be3eb18c1000607f7bfa8d3982b75500bc5c49d"
Jul 29 00:42:26.148713 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.142523    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"]
Jul 29 00:42:26.148713 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.147382    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9d8ec5a2-f75f4685d-qnlmc"]
Jul 29 00:42:26.153155 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.153128    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"]
Jul 29 00:42:26.155184 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.155164    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9d8ec5a2-7b6f9c7b5b-t4f8z"]
Jul 29 00:42:26.162892 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.162868    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"]
Jul 29 00:42:26.169045 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:26.169026    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d8ec5a2-54f7cd6f9b-bkmrp"]
Jul 29 00:42:27.941169 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:27.941089    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06ad6907-fdf7-46fc-bee8-25ddf92364c4" path="/var/lib/kubelet/pods/06ad6907-fdf7-46fc-bee8-25ddf92364c4/volumes"
Jul 29 00:42:27.941540 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:27.941442    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42826e41-2abe-45d1-82e4-1eaaff02fce3" path="/var/lib/kubelet/pods/42826e41-2abe-45d1-82e4-1eaaff02fce3/volumes"
Jul 29 00:42:27.941772 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:42:27.941758    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2" path="/var/lib/kubelet/pods/fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2/volumes"
Jul 29 00:43:05.932148 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:43:05.932117    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:43:05.934782 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:43:05.934759    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:44:13.908006 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.907427    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"]
Jul 29 00:44:13.908617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.908369    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2" containerName="payload-processing"
Jul 29 00:44:13.908617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.908390    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2" containerName="payload-processing"
Jul 29 00:44:13.908617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.908405    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ad6907-fdf7-46fc-bee8-25ddf92364c4" containerName="maas-api"
Jul 29 00:44:13.908617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.908414    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ad6907-fdf7-46fc-bee8-25ddf92364c4" containerName="maas-api"
Jul 29 00:44:13.908617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.908436    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42826e41-2abe-45d1-82e4-1eaaff02fce3" containerName="payload-pre-processing"
Jul 29 00:44:13.908617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.908445    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42826e41-2abe-45d1-82e4-1eaaff02fce3" containerName="payload-pre-processing"
Jul 29 00:44:13.908617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.908576    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd2c92b5-8c84-4254-9d4d-d4e6ba3f7dd2" containerName="payload-processing"
Jul 29 00:44:13.908617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.908595    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42826e41-2abe-45d1-82e4-1eaaff02fce3" containerName="payload-pre-processing"
Jul 29 00:44:13.908617 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.908613    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="06ad6907-fdf7-46fc-bee8-25ddf92364c4" containerName="maas-api"
Jul 29 00:44:13.915378 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.914899    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:13.917176 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.917151    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-065b9c-serving-cert\""
Jul 29 00:44:13.924129 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.924104    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"]
Jul 29 00:44:13.986515 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.986479    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr89j\" (UniqueName: \"kubernetes.io/projected/42b95319-fae5-4642-ae58-5d340ff17c20-kube-api-access-dr89j\") pod \"maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:13.986707 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.986531    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42b95319-fae5-4642-ae58-5d340ff17c20-maas-api-tls\") pod \"maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:13.986707 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:13.986556    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42b95319-fae5-4642-ae58-5d340ff17c20-openshift-service-ca\") pod \"maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:14.087494 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.087457    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr89j\" (UniqueName: \"kubernetes.io/projected/42b95319-fae5-4642-ae58-5d340ff17c20-kube-api-access-dr89j\") pod \"maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:14.087697 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.087522    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42b95319-fae5-4642-ae58-5d340ff17c20-maas-api-tls\") pod \"maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:14.087697 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.087565    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42b95319-fae5-4642-ae58-5d340ff17c20-openshift-service-ca\") pod \"maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:14.087697 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:44:14.087669    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-serving-cert: secret "maas-api-e2e-inf-a-065b9c-serving-cert" not found
Jul 29 00:44:14.087889 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:44:14.087755    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b95319-fae5-4642-ae58-5d340ff17c20-maas-api-tls podName:42b95319-fae5-4642-ae58-5d340ff17c20 nodeName:}" failed. No retries permitted until 2026-07-29 00:44:14.587733017 +0000 UTC m=+2169.166430454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/42b95319-fae5-4642-ae58-5d340ff17c20-maas-api-tls") pod "maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8" (UID: "42b95319-fae5-4642-ae58-5d340ff17c20") : secret "maas-api-e2e-inf-a-065b9c-serving-cert" not found
Jul 29 00:44:14.088272 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.088252    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42b95319-fae5-4642-ae58-5d340ff17c20-openshift-service-ca\") pod \"maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:14.097283 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.097254    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr89j\" (UniqueName: \"kubernetes.io/projected/42b95319-fae5-4642-ae58-5d340ff17c20-kube-api-access-dr89j\") pod \"maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:14.593419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.593375    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42b95319-fae5-4642-ae58-5d340ff17c20-maas-api-tls\") pod \"maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:14.595939 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.595913    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42b95319-fae5-4642-ae58-5d340ff17c20-maas-api-tls\") pod \"maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:14.625446 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.625416    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"]
Jul 29 00:44:14.628796 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.628780    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:44:14.630546 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.630525    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-065b9c\""
Jul 29 00:44:14.635770 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.635743    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"]
Jul 29 00:44:14.688442 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.688410    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"]
Jul 29 00:44:14.691603 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.691583    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:44:14.693243 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.693221    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-065b9c-dockercfg-qbbsj\""
Jul 29 00:44:14.694643 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.694621    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29a96e42-e73e-4f0e-afea-57b96ef8b311-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn\" (UID: \"29a96e42-e73e-4f0e-afea-57b96ef8b311\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:44:14.694767 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.694714    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn5ng\" (UniqueName: \"kubernetes.io/projected/29a96e42-e73e-4f0e-afea-57b96ef8b311-kube-api-access-zn5ng\") pod \"payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn\" (UID: \"29a96e42-e73e-4f0e-afea-57b96ef8b311\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:44:14.700965 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.700940    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"]
Jul 29 00:44:14.795585 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.795544    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5xt8\" (UniqueName: \"kubernetes.io/projected/077a3bdf-2609-459e-b7ed-2ba96c464a01-kube-api-access-r5xt8\") pod \"payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d\" (UID: \"077a3bdf-2609-459e-b7ed-2ba96c464a01\") " pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:44:14.795811 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.795600    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/077a3bdf-2609-459e-b7ed-2ba96c464a01-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d\" (UID: \"077a3bdf-2609-459e-b7ed-2ba96c464a01\") " pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:44:14.795811 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.795746    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29a96e42-e73e-4f0e-afea-57b96ef8b311-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn\" (UID: \"29a96e42-e73e-4f0e-afea-57b96ef8b311\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:44:14.795942 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.795810    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn5ng\" (UniqueName: \"kubernetes.io/projected/29a96e42-e73e-4f0e-afea-57b96ef8b311-kube-api-access-zn5ng\") pod \"payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn\" (UID: \"29a96e42-e73e-4f0e-afea-57b96ef8b311\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:44:14.796760 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.796737    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29a96e42-e73e-4f0e-afea-57b96ef8b311-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn\" (UID: \"29a96e42-e73e-4f0e-afea-57b96ef8b311\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:44:14.803138 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.803114    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn5ng\" (UniqueName: \"kubernetes.io/projected/29a96e42-e73e-4f0e-afea-57b96ef8b311-kube-api-access-zn5ng\") pod \"payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn\" (UID: \"29a96e42-e73e-4f0e-afea-57b96ef8b311\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:44:14.833527 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.833476    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:14.896832 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.896780    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5xt8\" (UniqueName: \"kubernetes.io/projected/077a3bdf-2609-459e-b7ed-2ba96c464a01-kube-api-access-r5xt8\") pod \"payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d\" (UID: \"077a3bdf-2609-459e-b7ed-2ba96c464a01\") " pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:44:14.897001 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.896847    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/077a3bdf-2609-459e-b7ed-2ba96c464a01-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d\" (UID: \"077a3bdf-2609-459e-b7ed-2ba96c464a01\") " pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:44:14.897721 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.897696    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/077a3bdf-2609-459e-b7ed-2ba96c464a01-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d\" (UID: \"077a3bdf-2609-459e-b7ed-2ba96c464a01\") " pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:44:14.904095 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.904069    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5xt8\" (UniqueName: \"kubernetes.io/projected/077a3bdf-2609-459e-b7ed-2ba96c464a01-kube-api-access-r5xt8\") pod \"payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d\" (UID: \"077a3bdf-2609-459e-b7ed-2ba96c464a01\") " pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:44:14.940675 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.940630    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:44:14.972418 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:14.972340    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"]
Jul 29 00:44:14.975901 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:44:14.975858    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b95319_fae5_4642_ae58_5d340ff17c20.slice/crio-7ab108bf2b41da625d4c72a3bdae9b630f537c79589b0f648e71116d82ea8c7f WatchSource:0}: Error finding container 7ab108bf2b41da625d4c72a3bdae9b630f537c79589b0f648e71116d82ea8c7f: Status 404 returned error can't find the container with id 7ab108bf2b41da625d4c72a3bdae9b630f537c79589b0f648e71116d82ea8c7f
Jul 29 00:44:15.003299 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.003271    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:44:15.097723 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.097698    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"]
Jul 29 00:44:15.099522 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:44:15.099486    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a96e42_e73e_4f0e_afea_57b96ef8b311.slice/crio-8a9aeb32f824db8666a4f3cf1ab6e67775ae34f12728cb872553a624ce599b82 WatchSource:0}: Error finding container 8a9aeb32f824db8666a4f3cf1ab6e67775ae34f12728cb872553a624ce599b82: Status 404 returned error can't find the container with id 8a9aeb32f824db8666a4f3cf1ab6e67775ae34f12728cb872553a624ce599b82
Jul 29 00:44:15.364907 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.364833    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"]
Jul 29 00:44:15.368116 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:44:15.368087    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077a3bdf_2609_459e_b7ed_2ba96c464a01.slice/crio-0d490d570a77f71411c42027ada1b4fe27f23b218df2297f37c73aa93b76ac36 WatchSource:0}: Error finding container 0d490d570a77f71411c42027ada1b4fe27f23b218df2297f37c73aa93b76ac36: Status 404 returned error can't find the container with id 0d490d570a77f71411c42027ada1b4fe27f23b218df2297f37c73aa93b76ac36
Jul 29 00:44:15.617718 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.617582    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d" event={"ID":"077a3bdf-2609-459e-b7ed-2ba96c464a01","Type":"ContainerStarted","Data":"64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934"}
Jul 29 00:44:15.617718 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.617680    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d" event={"ID":"077a3bdf-2609-459e-b7ed-2ba96c464a01","Type":"ContainerStarted","Data":"0d490d570a77f71411c42027ada1b4fe27f23b218df2297f37c73aa93b76ac36"}
Jul 29 00:44:15.617718 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.617715    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:44:15.619271 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.619242    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn" event={"ID":"29a96e42-e73e-4f0e-afea-57b96ef8b311","Type":"ContainerStarted","Data":"156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4"}
Jul 29 00:44:15.619395 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.619292    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn" event={"ID":"29a96e42-e73e-4f0e-afea-57b96ef8b311","Type":"ContainerStarted","Data":"8a9aeb32f824db8666a4f3cf1ab6e67775ae34f12728cb872553a624ce599b82"}
Jul 29 00:44:15.619395 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.619333    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:44:15.621204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.621175    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8" event={"ID":"42b95319-fae5-4642-ae58-5d340ff17c20","Type":"ContainerStarted","Data":"6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72"}
Jul 29 00:44:15.621314 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.621225    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8" event={"ID":"42b95319-fae5-4642-ae58-5d340ff17c20","Type":"ContainerStarted","Data":"7ab108bf2b41da625d4c72a3bdae9b630f537c79589b0f648e71116d82ea8c7f"}
Jul 29 00:44:15.621314 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.621248    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:15.634519 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.634457    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d" podStartSLOduration=1.634425554 podStartE2EDuration="1.634425554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:44:15.632569274 +0000 UTC m=+2170.211266718" watchObservedRunningTime="2026-07-29 00:44:15.634425554 +0000 UTC m=+2170.213122977"
Jul 29 00:44:15.646793 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.646748    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8" podStartSLOduration=2.23486481 podStartE2EDuration="2.646732626s" totalImagesPullingTime="411.867816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:44:15.645774817 +0000 UTC m=+2170.224472263" watchObservedRunningTime="2026-07-29 00:44:15.646732626 +0000 UTC m=+2170.225430070"
Jul 29 00:44:15.661947 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:15.661895    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn" podStartSLOduration=1.661880181 podStartE2EDuration="1.661880181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:44:15.660124029 +0000 UTC m=+2170.238821477" watchObservedRunningTime="2026-07-29 00:44:15.661880181 +0000 UTC m=+2170.240577626"
Jul 29 00:44:21.631980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:21.631950    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:44:26.627274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:26.627240    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:44:26.627703 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:26.627317    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:44:41.896591 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:41.896550    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"]
Jul 29 00:44:41.900105 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:41.900084    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:41.901880 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:41.901854    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ad97f4-serving-cert\""
Jul 29 00:44:41.909003 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:41.908979    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"]
Jul 29 00:44:41.945110 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:41.945069    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:41.945110 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:41.945113    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbdql\" (UniqueName: \"kubernetes.io/projected/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-kube-api-access-kbdql\") pod \"maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:41.945346 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:41.945183    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-maas-api-tls\") pod \"maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:42.046560 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.046528    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:42.046560 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.046567    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbdql\" (UniqueName: \"kubernetes.io/projected/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-kube-api-access-kbdql\") pod \"maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:42.046864 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.046615    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-maas-api-tls\") pod \"maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:42.046864 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:44:42.046775    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-serving-cert: secret "maas-api-e2e-inf-b-ad97f4-serving-cert" not found
Jul 29 00:44:42.046864 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:44:42.046838    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-maas-api-tls podName:bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e nodeName:}" failed. No retries permitted until 2026-07-29 00:44:42.54681748 +0000 UTC m=+2197.125514904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-maas-api-tls") pod "maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx" (UID: "bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e") : secret "maas-api-e2e-inf-b-ad97f4-serving-cert" not found
Jul 29 00:44:42.047350 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.047331    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:42.054389 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.054363    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbdql\" (UniqueName: \"kubernetes.io/projected/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-kube-api-access-kbdql\") pod \"maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:42.550090 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.550048    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-maas-api-tls\") pod \"maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:42.552524 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.552496    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-maas-api-tls\") pod \"maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:42.707084 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.707051    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"]
Jul 29 00:44:42.710636 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.710615    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:44:42.712310 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.712289    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ad97f4-dockercfg-w6t6f\""
Jul 29 00:44:42.712445 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.712421    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ad97f4\""
Jul 29 00:44:42.723629 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.723562    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"]
Jul 29 00:44:42.752107 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.752072    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxslk\" (UniqueName: \"kubernetes.io/projected/9f1440b2-ae53-4908-a334-81593e78dcb3-kube-api-access-nxslk\") pod \"payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh\" (UID: \"9f1440b2-ae53-4908-a334-81593e78dcb3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:44:42.752255 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.752131    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f1440b2-ae53-4908-a334-81593e78dcb3-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh\" (UID: \"9f1440b2-ae53-4908-a334-81593e78dcb3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:44:42.813742 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.813627    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:42.853321 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.853278    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxslk\" (UniqueName: \"kubernetes.io/projected/9f1440b2-ae53-4908-a334-81593e78dcb3-kube-api-access-nxslk\") pod \"payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh\" (UID: \"9f1440b2-ae53-4908-a334-81593e78dcb3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:44:42.853509 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.853340    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f1440b2-ae53-4908-a334-81593e78dcb3-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh\" (UID: \"9f1440b2-ae53-4908-a334-81593e78dcb3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:44:42.854186 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.854162    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f1440b2-ae53-4908-a334-81593e78dcb3-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh\" (UID: \"9f1440b2-ae53-4908-a334-81593e78dcb3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:44:42.861262 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.861230    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxslk\" (UniqueName: \"kubernetes.io/projected/9f1440b2-ae53-4908-a334-81593e78dcb3-kube-api-access-nxslk\") pod \"payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh\" (UID: \"9f1440b2-ae53-4908-a334-81593e78dcb3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:44:42.962389 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:42.962356    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"]
Jul 29 00:44:42.965603 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:44:42.965577    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf6b41e8_5fe8_47a7_b4ed_5821affc8a4e.slice/crio-e687a86a1127db5f577aefd59737a0ac50e32f942a7b33550a3ada7c1c7778ab WatchSource:0}: Error finding container e687a86a1127db5f577aefd59737a0ac50e32f942a7b33550a3ada7c1c7778ab: Status 404 returned error can't find the container with id e687a86a1127db5f577aefd59737a0ac50e32f942a7b33550a3ada7c1c7778ab
Jul 29 00:44:43.021855 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:43.021824    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:44:43.161493 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:43.161449    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"]
Jul 29 00:44:43.162755 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:44:43.162732    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1440b2_ae53_4908_a334_81593e78dcb3.slice/crio-c7355f5165a1da61dae4298d16cd15782604fca791ea0a6ea293d73b4491a5fd WatchSource:0}: Error finding container c7355f5165a1da61dae4298d16cd15782604fca791ea0a6ea293d73b4491a5fd: Status 404 returned error can't find the container with id c7355f5165a1da61dae4298d16cd15782604fca791ea0a6ea293d73b4491a5fd
Jul 29 00:44:43.745907 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:43.745866    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh" event={"ID":"9f1440b2-ae53-4908-a334-81593e78dcb3","Type":"ContainerStarted","Data":"4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376"}
Jul 29 00:44:43.746108 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:43.745923    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh" event={"ID":"9f1440b2-ae53-4908-a334-81593e78dcb3","Type":"ContainerStarted","Data":"c7355f5165a1da61dae4298d16cd15782604fca791ea0a6ea293d73b4491a5fd"}
Jul 29 00:44:43.746108 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:43.745979    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:44:43.747440 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:43.747419    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx" event={"ID":"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e","Type":"ContainerStarted","Data":"5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574"}
Jul 29 00:44:43.747557 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:43.747454    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx" event={"ID":"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e","Type":"ContainerStarted","Data":"e687a86a1127db5f577aefd59737a0ac50e32f942a7b33550a3ada7c1c7778ab"}
Jul 29 00:44:43.747557 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:43.747543    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:43.763275 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:43.763218    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh" podStartSLOduration=1.763200552 podStartE2EDuration="1.763200552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:44:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:44:43.762013932 +0000 UTC m=+2198.340711377" watchObservedRunningTime="2026-07-29 00:44:43.763200552 +0000 UTC m=+2198.341897999"
Jul 29 00:44:43.777909 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:43.777854    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx" podStartSLOduration=2.406651277 podStartE2EDuration="2.77783323s" totalImagesPullingTime="371.181953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:44:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:44:43.77534674 +0000 UTC m=+2198.354044188" watchObservedRunningTime="2026-07-29 00:44:43.77783323 +0000 UTC m=+2198.356530681"
Jul 29 00:44:49.758753 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:49.758723    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:44:54.755634 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:44:54.755606    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:46:53.631919 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:53.631870    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"]
Jul 29 00:46:53.632563 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:53.632141    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8" podUID="42b95319-fae5-4642-ae58-5d340ff17c20" containerName="maas-api" containerID="cri-o://6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72" gracePeriod=30
Jul 29 00:46:53.687257 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:53.687220    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"]
Jul 29 00:46:53.687509 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:53.687481    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d" podUID="077a3bdf-2609-459e-b7ed-2ba96c464a01" containerName="payload-processing" containerID="cri-o://64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934" gracePeriod=30
Jul 29 00:46:53.695248 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:53.695155    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"]
Jul 29 00:46:53.695863 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:53.695388    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn" podUID="29a96e42-e73e-4f0e-afea-57b96ef8b311" containerName="payload-pre-processing" containerID="cri-o://156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4" gracePeriod=30
Jul 29 00:46:53.991871 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:53.991840    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:46:53.995201 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:53.995174    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:46:54.076280 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.076253    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:46:54.137608 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.137565    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn5ng\" (UniqueName: \"kubernetes.io/projected/29a96e42-e73e-4f0e-afea-57b96ef8b311-kube-api-access-zn5ng\") pod \"29a96e42-e73e-4f0e-afea-57b96ef8b311\" (UID: \"29a96e42-e73e-4f0e-afea-57b96ef8b311\") "
Jul 29 00:46:54.137810 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.137621    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29a96e42-e73e-4f0e-afea-57b96ef8b311-plugins-config-volume\") pod \"29a96e42-e73e-4f0e-afea-57b96ef8b311\" (UID: \"29a96e42-e73e-4f0e-afea-57b96ef8b311\") "
Jul 29 00:46:54.137810 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.137787    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5xt8\" (UniqueName: \"kubernetes.io/projected/077a3bdf-2609-459e-b7ed-2ba96c464a01-kube-api-access-r5xt8\") pod \"077a3bdf-2609-459e-b7ed-2ba96c464a01\" (UID: \"077a3bdf-2609-459e-b7ed-2ba96c464a01\") "
Jul 29 00:46:54.137907 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.137813    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/077a3bdf-2609-459e-b7ed-2ba96c464a01-plugins-config-volume\") pod \"077a3bdf-2609-459e-b7ed-2ba96c464a01\" (UID: \"077a3bdf-2609-459e-b7ed-2ba96c464a01\") "
Jul 29 00:46:54.138055 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.138023    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29a96e42-e73e-4f0e-afea-57b96ef8b311-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "29a96e42-e73e-4f0e-afea-57b96ef8b311" (UID: "29a96e42-e73e-4f0e-afea-57b96ef8b311"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:46:54.138274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.138250    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/077a3bdf-2609-459e-b7ed-2ba96c464a01-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "077a3bdf-2609-459e-b7ed-2ba96c464a01" (UID: "077a3bdf-2609-459e-b7ed-2ba96c464a01"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:46:54.138274 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.138264    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29a96e42-e73e-4f0e-afea-57b96ef8b311-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:46:54.139877 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.139851    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/077a3bdf-2609-459e-b7ed-2ba96c464a01-kube-api-access-r5xt8" (OuterVolumeSpecName: "kube-api-access-r5xt8") pod "077a3bdf-2609-459e-b7ed-2ba96c464a01" (UID: "077a3bdf-2609-459e-b7ed-2ba96c464a01"). InnerVolumeSpecName "kube-api-access-r5xt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:46:54.139972 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.139884    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a96e42-e73e-4f0e-afea-57b96ef8b311-kube-api-access-zn5ng" (OuterVolumeSpecName: "kube-api-access-zn5ng") pod "29a96e42-e73e-4f0e-afea-57b96ef8b311" (UID: "29a96e42-e73e-4f0e-afea-57b96ef8b311"). InnerVolumeSpecName "kube-api-access-zn5ng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:46:54.238739 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.238628    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42b95319-fae5-4642-ae58-5d340ff17c20-maas-api-tls\") pod \"42b95319-fae5-4642-ae58-5d340ff17c20\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") "
Jul 29 00:46:54.238739 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.238729    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr89j\" (UniqueName: \"kubernetes.io/projected/42b95319-fae5-4642-ae58-5d340ff17c20-kube-api-access-dr89j\") pod \"42b95319-fae5-4642-ae58-5d340ff17c20\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") "
Jul 29 00:46:54.238930 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.238760    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42b95319-fae5-4642-ae58-5d340ff17c20-openshift-service-ca\") pod \"42b95319-fae5-4642-ae58-5d340ff17c20\" (UID: \"42b95319-fae5-4642-ae58-5d340ff17c20\") "
Jul 29 00:46:54.239049 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.239028    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5xt8\" (UniqueName: \"kubernetes.io/projected/077a3bdf-2609-459e-b7ed-2ba96c464a01-kube-api-access-r5xt8\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:46:54.239089 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.239057    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/077a3bdf-2609-459e-b7ed-2ba96c464a01-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:46:54.239089 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.239072    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn5ng\" (UniqueName: \"kubernetes.io/projected/29a96e42-e73e-4f0e-afea-57b96ef8b311-kube-api-access-zn5ng\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:46:54.239178 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.239160    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42b95319-fae5-4642-ae58-5d340ff17c20-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42b95319-fae5-4642-ae58-5d340ff17c20" (UID: "42b95319-fae5-4642-ae58-5d340ff17c20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:46:54.240778 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.240754    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42b95319-fae5-4642-ae58-5d340ff17c20-kube-api-access-dr89j" (OuterVolumeSpecName: "kube-api-access-dr89j") pod "42b95319-fae5-4642-ae58-5d340ff17c20" (UID: "42b95319-fae5-4642-ae58-5d340ff17c20"). InnerVolumeSpecName "kube-api-access-dr89j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:46:54.240842 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.240788    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42b95319-fae5-4642-ae58-5d340ff17c20-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42b95319-fae5-4642-ae58-5d340ff17c20" (UID: "42b95319-fae5-4642-ae58-5d340ff17c20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:46:54.339516 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.339487    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42b95319-fae5-4642-ae58-5d340ff17c20-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:46:54.339516 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.339514    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr89j\" (UniqueName: \"kubernetes.io/projected/42b95319-fae5-4642-ae58-5d340ff17c20-kube-api-access-dr89j\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:46:54.339759 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.339527    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42b95319-fae5-4642-ae58-5d340ff17c20-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:46:54.358239 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.358203    2557 generic.go:356] "Generic (PLEG): container finished" podID="077a3bdf-2609-459e-b7ed-2ba96c464a01" containerID="64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934" exitCode=0
Jul 29 00:46:54.358419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.358270    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"
Jul 29 00:46:54.358419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.358283    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d" event={"ID":"077a3bdf-2609-459e-b7ed-2ba96c464a01","Type":"ContainerDied","Data":"64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934"}
Jul 29 00:46:54.358419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.358375    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d" event={"ID":"077a3bdf-2609-459e-b7ed-2ba96c464a01","Type":"ContainerDied","Data":"0d490d570a77f71411c42027ada1b4fe27f23b218df2297f37c73aa93b76ac36"}
Jul 29 00:46:54.358419 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.358408    2557 scope.go:117] "RemoveContainer" containerID="64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934"
Jul 29 00:46:54.359621 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.359601    2557 generic.go:356] "Generic (PLEG): container finished" podID="29a96e42-e73e-4f0e-afea-57b96ef8b311" containerID="156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4" exitCode=0
Jul 29 00:46:54.359738 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.359635    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn" event={"ID":"29a96e42-e73e-4f0e-afea-57b96ef8b311","Type":"ContainerDied","Data":"156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4"}
Jul 29 00:46:54.359738 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.359685    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"
Jul 29 00:46:54.359738 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.359705    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn" event={"ID":"29a96e42-e73e-4f0e-afea-57b96ef8b311","Type":"ContainerDied","Data":"8a9aeb32f824db8666a4f3cf1ab6e67775ae34f12728cb872553a624ce599b82"}
Jul 29 00:46:54.361075 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.361048    2557 generic.go:356] "Generic (PLEG): container finished" podID="42b95319-fae5-4642-ae58-5d340ff17c20" containerID="6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72" exitCode=0
Jul 29 00:46:54.361187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.361117    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8" event={"ID":"42b95319-fae5-4642-ae58-5d340ff17c20","Type":"ContainerDied","Data":"6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72"}
Jul 29 00:46:54.361187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.361161    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8" event={"ID":"42b95319-fae5-4642-ae58-5d340ff17c20","Type":"ContainerDied","Data":"7ab108bf2b41da625d4c72a3bdae9b630f537c79589b0f648e71116d82ea8c7f"}
Jul 29 00:46:54.361187 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.361165    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"
Jul 29 00:46:54.368265 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.368244    2557 scope.go:117] "RemoveContainer" containerID="64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934"
Jul 29 00:46:54.368568 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:46:54.368546    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934\": container with ID starting with 64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934 not found: ID does not exist" containerID="64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934"
Jul 29 00:46:54.368695 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.368575    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934"} err="failed to get container status \"64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934\": rpc error: code = NotFound desc = could not find container \"64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934\": container with ID starting with 64c98c045895c3d4a963ebec9165e298d9f765a2c5c9b467b292cc851fc7a934 not found: ID does not exist"
Jul 29 00:46:54.368695 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.368593    2557 scope.go:117] "RemoveContainer" containerID="156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4"
Jul 29 00:46:54.377601 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.377582    2557 scope.go:117] "RemoveContainer" containerID="156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4"
Jul 29 00:46:54.377883 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:46:54.377861    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4\": container with ID starting with 156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4 not found: ID does not exist" containerID="156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4"
Jul 29 00:46:54.377996 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.377890    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4"} err="failed to get container status \"156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4\": rpc error: code = NotFound desc = could not find container \"156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4\": container with ID starting with 156bfeeef97d7e9ee533fc75fc5f9c486835c715eebf499eb01875ac9ffb37e4 not found: ID does not exist"
Jul 29 00:46:54.377996 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.377907    2557 scope.go:117] "RemoveContainer" containerID="6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72"
Jul 29 00:46:54.384514 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.384489    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"]
Jul 29 00:46:54.386090 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.386069    2557 scope.go:117] "RemoveContainer" containerID="6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72"
Jul 29 00:46:54.386530 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:46:54.386505    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72\": container with ID starting with 6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72 not found: ID does not exist" containerID="6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72"
Jul 29 00:46:54.386645 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.386604    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72"} err="failed to get container status \"6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72\": rpc error: code = NotFound desc = could not find container \"6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72\": container with ID starting with 6111c84c1c8d38d06ca4dec304d0efb938dbede3ed1a5d640db7a5ae8440fd72 not found: ID does not exist"
Jul 29 00:46:54.388231 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.388209    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-065b9c-5976497596-4xn5d"]
Jul 29 00:46:54.396124 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.396101    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"]
Jul 29 00:46:54.399281 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.399260    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-065b9c-5485d4c454-j4sl8"]
Jul 29 00:46:54.407041 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.407014    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"]
Jul 29 00:46:54.411043 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:54.411019    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-065b9c-5bb896ddbd-tkgxn"]
Jul 29 00:46:55.941180 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:55.941147    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="077a3bdf-2609-459e-b7ed-2ba96c464a01" path="/var/lib/kubelet/pods/077a3bdf-2609-459e-b7ed-2ba96c464a01/volumes"
Jul 29 00:46:55.941568 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:55.941504    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29a96e42-e73e-4f0e-afea-57b96ef8b311" path="/var/lib/kubelet/pods/29a96e42-e73e-4f0e-afea-57b96ef8b311/volumes"
Jul 29 00:46:55.941868 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:46:55.941854    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42b95319-fae5-4642-ae58-5d340ff17c20" path="/var/lib/kubelet/pods/42b95319-fae5-4642-ae58-5d340ff17c20/volumes"
Jul 29 00:47:11.001035 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.001005    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"]
Jul 29 00:47:11.001548 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.001217    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx" podUID="bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e" containerName="maas-api" containerID="cri-o://5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574" gracePeriod=30
Jul 29 00:47:11.051072 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.051040    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"]
Jul 29 00:47:11.051337 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.051299    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh" podUID="9f1440b2-ae53-4908-a334-81593e78dcb3" containerName="payload-processing" containerID="cri-o://4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376" gracePeriod=30
Jul 29 00:47:11.313290 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.312976    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:47:11.354206 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.354182    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:47:11.372223 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.372203    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-openshift-service-ca\") pod \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") "
Jul 29 00:47:11.372330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.372262    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f1440b2-ae53-4908-a334-81593e78dcb3-plugins-config-volume\") pod \"9f1440b2-ae53-4908-a334-81593e78dcb3\" (UID: \"9f1440b2-ae53-4908-a334-81593e78dcb3\") "
Jul 29 00:47:11.372330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.372303    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-maas-api-tls\") pod \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") "
Jul 29 00:47:11.372330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.372329    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxslk\" (UniqueName: \"kubernetes.io/projected/9f1440b2-ae53-4908-a334-81593e78dcb3-kube-api-access-nxslk\") pod \"9f1440b2-ae53-4908-a334-81593e78dcb3\" (UID: \"9f1440b2-ae53-4908-a334-81593e78dcb3\") "
Jul 29 00:47:11.372462 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.372347    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbdql\" (UniqueName: \"kubernetes.io/projected/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-kube-api-access-kbdql\") pod \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\" (UID: \"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e\") "
Jul 29 00:47:11.372608 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.372579    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e" (UID: "bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:47:11.372756 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.372727    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f1440b2-ae53-4908-a334-81593e78dcb3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f1440b2-ae53-4908-a334-81593e78dcb3" (UID: "9f1440b2-ae53-4908-a334-81593e78dcb3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:47:11.375101 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.375075    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-kube-api-access-kbdql" (OuterVolumeSpecName: "kube-api-access-kbdql") pod "bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e" (UID: "bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e"). InnerVolumeSpecName "kube-api-access-kbdql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:47:11.375101 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.375084    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e" (UID: "bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:47:11.375231 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.375175    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f1440b2-ae53-4908-a334-81593e78dcb3-kube-api-access-nxslk" (OuterVolumeSpecName: "kube-api-access-nxslk") pod "9f1440b2-ae53-4908-a334-81593e78dcb3" (UID: "9f1440b2-ae53-4908-a334-81593e78dcb3"). InnerVolumeSpecName "kube-api-access-nxslk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:47:11.450603 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.450567    2557 generic.go:356] "Generic (PLEG): container finished" podID="9f1440b2-ae53-4908-a334-81593e78dcb3" containerID="4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376" exitCode=0
Jul 29 00:47:11.450819 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.450636    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"
Jul 29 00:47:11.450819 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.450673    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh" event={"ID":"9f1440b2-ae53-4908-a334-81593e78dcb3","Type":"ContainerDied","Data":"4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376"}
Jul 29 00:47:11.450819 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.450772    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh" event={"ID":"9f1440b2-ae53-4908-a334-81593e78dcb3","Type":"ContainerDied","Data":"c7355f5165a1da61dae4298d16cd15782604fca791ea0a6ea293d73b4491a5fd"}
Jul 29 00:47:11.450819 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.450802    2557 scope.go:117] "RemoveContainer" containerID="4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376"
Jul 29 00:47:11.452412 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.452374    2557 generic.go:356] "Generic (PLEG): container finished" podID="bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e" containerID="5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574" exitCode=0
Jul 29 00:47:11.452548 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.452417    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx" event={"ID":"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e","Type":"ContainerDied","Data":"5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574"}
Jul 29 00:47:11.452548 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.452439    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"
Jul 29 00:47:11.452548 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.452454    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx" event={"ID":"bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e","Type":"ContainerDied","Data":"e687a86a1127db5f577aefd59737a0ac50e32f942a7b33550a3ada7c1c7778ab"}
Jul 29 00:47:11.460006 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.459989    2557 scope.go:117] "RemoveContainer" containerID="4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376"
Jul 29 00:47:11.460291 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:47:11.460260    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376\": container with ID starting with 4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376 not found: ID does not exist" containerID="4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376"
Jul 29 00:47:11.460410 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.460301    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376"} err="failed to get container status \"4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376\": rpc error: code = NotFound desc = could not find container \"4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376\": container with ID starting with 4b2c6555da8fd61b793190e70650e7ab7922efa15302aa231cd71d0071c48376 not found: ID does not exist"
Jul 29 00:47:11.460410 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.460324    2557 scope.go:117] "RemoveContainer" containerID="5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574"
Jul 29 00:47:11.468080 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.468064    2557 scope.go:117] "RemoveContainer" containerID="5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574"
Jul 29 00:47:11.468334 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:47:11.468313    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574\": container with ID starting with 5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574 not found: ID does not exist" containerID="5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574"
Jul 29 00:47:11.468416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.468339    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574"} err="failed to get container status \"5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574\": rpc error: code = NotFound desc = could not find container \"5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574\": container with ID starting with 5f78ed7894b29b1f749f846534d39aa80a99524716d03ebfd193a47fb5d68574 not found: ID does not exist"
Jul 29 00:47:11.472931 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.472907    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f1440b2-ae53-4908-a334-81593e78dcb3-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:47:11.473416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.472935    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:47:11.473416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.472950    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxslk\" (UniqueName: \"kubernetes.io/projected/9f1440b2-ae53-4908-a334-81593e78dcb3-kube-api-access-nxslk\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:47:11.473416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.472963    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbdql\" (UniqueName: \"kubernetes.io/projected/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-kube-api-access-kbdql\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:47:11.473416 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.472976    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:47:11.475221 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.475199    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"]
Jul 29 00:47:11.479811 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.479789    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad97f4-9bc7cc787-ms8xx"]
Jul 29 00:47:11.489062 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.489041    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"]
Jul 29 00:47:11.491192 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.491172    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ad97f4-657f79bfcd-54xlh"]
Jul 29 00:47:11.940636 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.940604    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f1440b2-ae53-4908-a334-81593e78dcb3" path="/var/lib/kubelet/pods/9f1440b2-ae53-4908-a334-81593e78dcb3/volumes"
Jul 29 00:47:11.941030 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:11.941014    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e" path="/var/lib/kubelet/pods/bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e/volumes"
Jul 29 00:47:21.916126 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916092    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"]
Jul 29 00:47:21.916676 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916582    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f1440b2-ae53-4908-a334-81593e78dcb3" containerName="payload-processing"
Jul 29 00:47:21.916676 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916602    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f1440b2-ae53-4908-a334-81593e78dcb3" containerName="payload-processing"
Jul 29 00:47:21.916676 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916625    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="077a3bdf-2609-459e-b7ed-2ba96c464a01" containerName="payload-processing"
Jul 29 00:47:21.916676 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916634    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="077a3bdf-2609-459e-b7ed-2ba96c464a01" containerName="payload-processing"
Jul 29 00:47:21.916676 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916646    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a96e42-e73e-4f0e-afea-57b96ef8b311" containerName="payload-pre-processing"
Jul 29 00:47:21.916676 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916670    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a96e42-e73e-4f0e-afea-57b96ef8b311" containerName="payload-pre-processing"
Jul 29 00:47:21.916992 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916689    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42b95319-fae5-4642-ae58-5d340ff17c20" containerName="maas-api"
Jul 29 00:47:21.916992 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916697    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42b95319-fae5-4642-ae58-5d340ff17c20" containerName="maas-api"
Jul 29 00:47:21.916992 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916706    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e" containerName="maas-api"
Jul 29 00:47:21.916992 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916713    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e" containerName="maas-api"
Jul 29 00:47:21.916992 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916807    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42b95319-fae5-4642-ae58-5d340ff17c20" containerName="maas-api"
Jul 29 00:47:21.916992 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916822    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f1440b2-ae53-4908-a334-81593e78dcb3" containerName="payload-processing"
Jul 29 00:47:21.916992 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916832    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="077a3bdf-2609-459e-b7ed-2ba96c464a01" containerName="payload-processing"
Jul 29 00:47:21.916992 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916840    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a96e42-e73e-4f0e-afea-57b96ef8b311" containerName="payload-pre-processing"
Jul 29 00:47:21.916992 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.916850    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf6b41e8-5fe8-47a7-b4ed-5821affc8a4e" containerName="maas-api"
Jul 29 00:47:21.920266 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.920246    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:21.922048 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.922028    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4f3846-serving-cert\""
Jul 29 00:47:21.932621 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.932597    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"]
Jul 29 00:47:21.959812 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.959419    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7376f235-af7e-4848-9c3e-44c0a1d56714-openshift-service-ca\") pod \"maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:21.959812 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.959467    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm9lk\" (UniqueName: \"kubernetes.io/projected/7376f235-af7e-4848-9c3e-44c0a1d56714-kube-api-access-mm9lk\") pod \"maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:21.959812 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:21.959602    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7376f235-af7e-4848-9c3e-44c0a1d56714-maas-api-tls\") pod \"maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:22.060314 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.060284    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7376f235-af7e-4848-9c3e-44c0a1d56714-openshift-service-ca\") pod \"maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:22.060481 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.060320    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm9lk\" (UniqueName: \"kubernetes.io/projected/7376f235-af7e-4848-9c3e-44c0a1d56714-kube-api-access-mm9lk\") pod \"maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:22.060481 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.060370    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7376f235-af7e-4848-9c3e-44c0a1d56714-maas-api-tls\") pod \"maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:22.060481 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:47:22.060469    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-serving-cert: secret "maas-api-e2e-api-a-4f3846-serving-cert" not found
Jul 29 00:47:22.060605 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:47:22.060532    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7376f235-af7e-4848-9c3e-44c0a1d56714-maas-api-tls podName:7376f235-af7e-4848-9c3e-44c0a1d56714 nodeName:}" failed. No retries permitted until 2026-07-29 00:47:22.560516145 +0000 UTC m=+2357.139213569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7376f235-af7e-4848-9c3e-44c0a1d56714-maas-api-tls") pod "maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9" (UID: "7376f235-af7e-4848-9c3e-44c0a1d56714") : secret "maas-api-e2e-api-a-4f3846-serving-cert" not found
Jul 29 00:47:22.060937 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.060916    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7376f235-af7e-4848-9c3e-44c0a1d56714-openshift-service-ca\") pod \"maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:22.070770 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.070749    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm9lk\" (UniqueName: \"kubernetes.io/projected/7376f235-af7e-4848-9c3e-44c0a1d56714-kube-api-access-mm9lk\") pod \"maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:22.566368 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.566332    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7376f235-af7e-4848-9c3e-44c0a1d56714-maas-api-tls\") pod \"maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:22.568887 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.568858    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7376f235-af7e-4848-9c3e-44c0a1d56714-maas-api-tls\") pod \"maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:22.685101 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.685069    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"]
Jul 29 00:47:22.688864 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.688843    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:47:22.691130 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.691110    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4f3846\""
Jul 29 00:47:22.691288 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.691271    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4f3846-dockercfg-bjrbz\""
Jul 29 00:47:22.701194 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.701169    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"]
Jul 29 00:47:22.768129 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.768095    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d6b9c1-d71a-4e77-810c-1617370792b2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4\" (UID: \"e6d6b9c1-d71a-4e77-810c-1617370792b2\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:47:22.768332 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.768160    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9wbj\" (UniqueName: \"kubernetes.io/projected/e6d6b9c1-d71a-4e77-810c-1617370792b2-kube-api-access-f9wbj\") pod \"payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4\" (UID: \"e6d6b9c1-d71a-4e77-810c-1617370792b2\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:47:22.833112 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.833024    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:22.869344 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.869309    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d6b9c1-d71a-4e77-810c-1617370792b2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4\" (UID: \"e6d6b9c1-d71a-4e77-810c-1617370792b2\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:47:22.869505 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.869400    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9wbj\" (UniqueName: \"kubernetes.io/projected/e6d6b9c1-d71a-4e77-810c-1617370792b2-kube-api-access-f9wbj\") pod \"payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4\" (UID: \"e6d6b9c1-d71a-4e77-810c-1617370792b2\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:47:22.870278 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.870253    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d6b9c1-d71a-4e77-810c-1617370792b2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4\" (UID: \"e6d6b9c1-d71a-4e77-810c-1617370792b2\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:47:22.877473 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.877450    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9wbj\" (UniqueName: \"kubernetes.io/projected/e6d6b9c1-d71a-4e77-810c-1617370792b2-kube-api-access-f9wbj\") pod \"payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4\" (UID: \"e6d6b9c1-d71a-4e77-810c-1617370792b2\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:47:22.975363 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.975341    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"]
Jul 29 00:47:22.978270 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:47:22.978241    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7376f235_af7e_4848_9c3e_44c0a1d56714.slice/crio-ae549f02fb9fcfc65d968641e23a60117a95ea6d21a47675faa5c5706d57185e WatchSource:0}: Error finding container ae549f02fb9fcfc65d968641e23a60117a95ea6d21a47675faa5c5706d57185e: Status 404 returned error can't find the container with id ae549f02fb9fcfc65d968641e23a60117a95ea6d21a47675faa5c5706d57185e
Jul 29 00:47:22.979671 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.979642    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:47:22.999053 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:22.999028    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:47:23.136296 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:23.136258    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"]
Jul 29 00:47:23.139465 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:47:23.139441    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d6b9c1_d71a_4e77_810c_1617370792b2.slice/crio-955b2661e323ae3e4960c30b224d2048a8477e3f9c4baef7b19f2202d601610c WatchSource:0}: Error finding container 955b2661e323ae3e4960c30b224d2048a8477e3f9c4baef7b19f2202d601610c: Status 404 returned error can't find the container with id 955b2661e323ae3e4960c30b224d2048a8477e3f9c4baef7b19f2202d601610c
Jul 29 00:47:23.511889 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:23.511855    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4" event={"ID":"e6d6b9c1-d71a-4e77-810c-1617370792b2","Type":"ContainerStarted","Data":"2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9"}
Jul 29 00:47:23.512091 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:23.511909    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4" event={"ID":"e6d6b9c1-d71a-4e77-810c-1617370792b2","Type":"ContainerStarted","Data":"955b2661e323ae3e4960c30b224d2048a8477e3f9c4baef7b19f2202d601610c"}
Jul 29 00:47:23.512091 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:23.512028    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:47:23.513029 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:23.513007    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9" event={"ID":"7376f235-af7e-4848-9c3e-44c0a1d56714","Type":"ContainerStarted","Data":"ae549f02fb9fcfc65d968641e23a60117a95ea6d21a47675faa5c5706d57185e"}
Jul 29 00:47:23.529450 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:23.529412    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4" podStartSLOduration=1.529399935 podStartE2EDuration="1.529399935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:47:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:47:23.528327459 +0000 UTC m=+2358.107024903" watchObservedRunningTime="2026-07-29 00:47:23.529399935 +0000 UTC m=+2358.108097382"
Jul 29 00:47:24.519585 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:24.519544    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9" event={"ID":"7376f235-af7e-4848-9c3e-44c0a1d56714","Type":"ContainerStarted","Data":"58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa"}
Jul 29 00:47:24.520052 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:24.519627    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:24.537433 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:24.537384    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9" podStartSLOduration=2.949361771 podStartE2EDuration="3.537370599s" totalImagesPullingTime="588.008828ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:47:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:47:24.535723684 +0000 UTC m=+2359.114421129" watchObservedRunningTime="2026-07-29 00:47:24.537370599 +0000 UTC m=+2359.116068071"
Jul 29 00:47:30.531742 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:30.531711    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:47:34.522436 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:34.522391    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:47:50.964558 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:50.964521    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"]
Jul 29 00:47:50.967909 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:50.967887    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:50.969874 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:50.969855    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-111f37-serving-cert\""
Jul 29 00:47:50.983494 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:50.983445    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"]
Jul 29 00:47:51.118267 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.118224    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-maas-api-tls\") pod \"maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:51.118489 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.118273    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqppg\" (UniqueName: \"kubernetes.io/projected/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-kube-api-access-pqppg\") pod \"maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:51.118489 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.118372    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-openshift-service-ca\") pod \"maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:51.219232 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.219116    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-maas-api-tls\") pod \"maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:51.219232 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.219172    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqppg\" (UniqueName: \"kubernetes.io/projected/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-kube-api-access-pqppg\") pod \"maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:51.219458 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.219264    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-openshift-service-ca\") pod \"maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:51.219458 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:47:51.219285    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-serving-cert: secret "maas-api-e2e-api-b-111f37-serving-cert" not found
Jul 29 00:47:51.219458 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:47:51.219356    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-maas-api-tls podName:5fd1ef26-d466-4c85-ad3a-8e60cea84e29 nodeName:}" failed. No retries permitted until 2026-07-29 00:47:51.719334878 +0000 UTC m=+2386.298032303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-maas-api-tls") pod "maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq" (UID: "5fd1ef26-d466-4c85-ad3a-8e60cea84e29") : secret "maas-api-e2e-api-b-111f37-serving-cert" not found
Jul 29 00:47:51.220060 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.220040    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-openshift-service-ca\") pod \"maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:51.229429 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.229398    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqppg\" (UniqueName: \"kubernetes.io/projected/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-kube-api-access-pqppg\") pod \"maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:51.725354 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.725321    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-maas-api-tls\") pod \"maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:51.727710 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.727653    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-maas-api-tls\") pod \"maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:51.880620 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:51.880586    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:52.020702 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:52.020677    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"]
Jul 29 00:47:52.024308 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:47:52.024276    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fd1ef26_d466_4c85_ad3a_8e60cea84e29.slice/crio-900ca0eb0d2aa992b40b6f5460890e21ca1f138ce2faeaa3272c10a70db36d60 WatchSource:0}: Error finding container 900ca0eb0d2aa992b40b6f5460890e21ca1f138ce2faeaa3272c10a70db36d60: Status 404 returned error can't find the container with id 900ca0eb0d2aa992b40b6f5460890e21ca1f138ce2faeaa3272c10a70db36d60
Jul 29 00:47:52.650677 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:52.650630    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq" event={"ID":"5fd1ef26-d466-4c85-ad3a-8e60cea84e29","Type":"ContainerStarted","Data":"197f5d8304459a56d9bb92fde3c8ed709de2e20c89417f85c5e5e8fb06ff768c"}
Jul 29 00:47:52.650861 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:52.650715    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq" event={"ID":"5fd1ef26-d466-4c85-ad3a-8e60cea84e29","Type":"ContainerStarted","Data":"900ca0eb0d2aa992b40b6f5460890e21ca1f138ce2faeaa3272c10a70db36d60"}
Jul 29 00:47:52.650935 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:52.650878    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:47:52.666846 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:52.666747    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq" podStartSLOduration=2.279280435 podStartE2EDuration="2.666730207s" totalImagesPullingTime="387.449772ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:47:52.664647767 +0000 UTC m=+2387.243345216" watchObservedRunningTime="2026-07-29 00:47:52.666730207 +0000 UTC m=+2387.245427652"
Jul 29 00:47:58.662937 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:47:58.662905    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:48:05.957841 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:05.957811    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:48:05.962996 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:05.962971    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:48:06.090089 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:06.090060    2557 scope.go:117] "RemoveContainer" containerID="2751117aad94284797367cf0132b2c5cf76f80c02a6c8552d48c96dcc9667ea1"
Jul 29 00:48:34.239643 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.239434    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"]
Jul 29 00:48:34.240232 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.239708    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9" podUID="7376f235-af7e-4848-9c3e-44c0a1d56714" containerName="maas-api" containerID="cri-o://58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa" gracePeriod=30
Jul 29 00:48:34.293473 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.293441    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"]
Jul 29 00:48:34.293754 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.293702    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4" podUID="e6d6b9c1-d71a-4e77-810c-1617370792b2" containerName="payload-processing" containerID="cri-o://2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9" gracePeriod=30
Jul 29 00:48:34.547525 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.547498    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:48:34.584521 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.584497    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:48:34.616607 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.616579    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm9lk\" (UniqueName: \"kubernetes.io/projected/7376f235-af7e-4848-9c3e-44c0a1d56714-kube-api-access-mm9lk\") pod \"7376f235-af7e-4848-9c3e-44c0a1d56714\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") "
Jul 29 00:48:34.616761 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.616632    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7376f235-af7e-4848-9c3e-44c0a1d56714-maas-api-tls\") pod \"7376f235-af7e-4848-9c3e-44c0a1d56714\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") "
Jul 29 00:48:34.616761 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.616727    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7376f235-af7e-4848-9c3e-44c0a1d56714-openshift-service-ca\") pod \"7376f235-af7e-4848-9c3e-44c0a1d56714\" (UID: \"7376f235-af7e-4848-9c3e-44c0a1d56714\") "
Jul 29 00:48:34.617065 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.617039    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7376f235-af7e-4848-9c3e-44c0a1d56714-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7376f235-af7e-4848-9c3e-44c0a1d56714" (UID: "7376f235-af7e-4848-9c3e-44c0a1d56714"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:48:34.618689 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.618634    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7376f235-af7e-4848-9c3e-44c0a1d56714-kube-api-access-mm9lk" (OuterVolumeSpecName: "kube-api-access-mm9lk") pod "7376f235-af7e-4848-9c3e-44c0a1d56714" (UID: "7376f235-af7e-4848-9c3e-44c0a1d56714"). InnerVolumeSpecName "kube-api-access-mm9lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:48:34.618689 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.618653    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7376f235-af7e-4848-9c3e-44c0a1d56714-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7376f235-af7e-4848-9c3e-44c0a1d56714" (UID: "7376f235-af7e-4848-9c3e-44c0a1d56714"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:48:34.717348 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.717311    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d6b9c1-d71a-4e77-810c-1617370792b2-plugins-config-volume\") pod \"e6d6b9c1-d71a-4e77-810c-1617370792b2\" (UID: \"e6d6b9c1-d71a-4e77-810c-1617370792b2\") "
Jul 29 00:48:34.717512 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.717388    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9wbj\" (UniqueName: \"kubernetes.io/projected/e6d6b9c1-d71a-4e77-810c-1617370792b2-kube-api-access-f9wbj\") pod \"e6d6b9c1-d71a-4e77-810c-1617370792b2\" (UID: \"e6d6b9c1-d71a-4e77-810c-1617370792b2\") "
Jul 29 00:48:34.717577 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.717563    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7376f235-af7e-4848-9c3e-44c0a1d56714-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:48:34.717621 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.717579    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm9lk\" (UniqueName: \"kubernetes.io/projected/7376f235-af7e-4848-9c3e-44c0a1d56714-kube-api-access-mm9lk\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:48:34.717621 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.717589    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7376f235-af7e-4848-9c3e-44c0a1d56714-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:48:34.717748 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.717716    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d6b9c1-d71a-4e77-810c-1617370792b2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6d6b9c1-d71a-4e77-810c-1617370792b2" (UID: "e6d6b9c1-d71a-4e77-810c-1617370792b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:48:34.719369 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.719349    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6d6b9c1-d71a-4e77-810c-1617370792b2-kube-api-access-f9wbj" (OuterVolumeSpecName: "kube-api-access-f9wbj") pod "e6d6b9c1-d71a-4e77-810c-1617370792b2" (UID: "e6d6b9c1-d71a-4e77-810c-1617370792b2"). InnerVolumeSpecName "kube-api-access-f9wbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:48:34.818065 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.817986    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d6b9c1-d71a-4e77-810c-1617370792b2-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:48:34.818065 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.818017    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9wbj\" (UniqueName: \"kubernetes.io/projected/e6d6b9c1-d71a-4e77-810c-1617370792b2-kube-api-access-f9wbj\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:48:34.849864 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.849831    2557 generic.go:356] "Generic (PLEG): container finished" podID="e6d6b9c1-d71a-4e77-810c-1617370792b2" containerID="2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9" exitCode=0
Jul 29 00:48:34.850032 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.849880    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"
Jul 29 00:48:34.850032 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.849923    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4" event={"ID":"e6d6b9c1-d71a-4e77-810c-1617370792b2","Type":"ContainerDied","Data":"2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9"}
Jul 29 00:48:34.850032 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.850021    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4" event={"ID":"e6d6b9c1-d71a-4e77-810c-1617370792b2","Type":"ContainerDied","Data":"955b2661e323ae3e4960c30b224d2048a8477e3f9c4baef7b19f2202d601610c"}
Jul 29 00:48:34.850220 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.850053    2557 scope.go:117] "RemoveContainer" containerID="2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9"
Jul 29 00:48:34.851859 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.851828    2557 generic.go:356] "Generic (PLEG): container finished" podID="7376f235-af7e-4848-9c3e-44c0a1d56714" containerID="58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa" exitCode=0
Jul 29 00:48:34.851980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.851897    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9" event={"ID":"7376f235-af7e-4848-9c3e-44c0a1d56714","Type":"ContainerDied","Data":"58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa"}
Jul 29 00:48:34.851980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.851938    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9" event={"ID":"7376f235-af7e-4848-9c3e-44c0a1d56714","Type":"ContainerDied","Data":"ae549f02fb9fcfc65d968641e23a60117a95ea6d21a47675faa5c5706d57185e"}
Jul 29 00:48:34.852097 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.852017    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"
Jul 29 00:48:34.858824 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.858805    2557 scope.go:117] "RemoveContainer" containerID="2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9"
Jul 29 00:48:34.859106 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:48:34.859082    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9\": container with ID starting with 2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9 not found: ID does not exist" containerID="2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9"
Jul 29 00:48:34.859204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.859111    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9"} err="failed to get container status \"2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9\": rpc error: code = NotFound desc = could not find container \"2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9\": container with ID starting with 2d63ba46a2c461e27bbb9aefc77e8e0d74ef71d99a0a01c455efeee01641b6e9 not found: ID does not exist"
Jul 29 00:48:34.859204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.859126    2557 scope.go:117] "RemoveContainer" containerID="58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa"
Jul 29 00:48:34.866927 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.866911    2557 scope.go:117] "RemoveContainer" containerID="58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa"
Jul 29 00:48:34.867181 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:48:34.867162    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa\": container with ID starting with 58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa not found: ID does not exist" containerID="58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa"
Jul 29 00:48:34.867241 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.867187    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa"} err="failed to get container status \"58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa\": rpc error: code = NotFound desc = could not find container \"58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa\": container with ID starting with 58c0f4def79824fa17e45da2fc0e25ab8900ed5d7f53dc5ea0d29de558fb8caa not found: ID does not exist"
Jul 29 00:48:34.873900 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.873879    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"]
Jul 29 00:48:34.877521 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.877500    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4"]
Jul 29 00:48:34.885197 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.885178    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"]
Jul 29 00:48:34.890978 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:34.890959    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f3846-cbcf56f7-bdnh9"]
Jul 29 00:48:35.521212 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:35.521117    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-api-a-4f3846-584ff55574-kkbj4" podUID="e6d6b9c1-d71a-4e77-810c-1617370792b2" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.63:9004: i/o timeout"
Jul 29 00:48:35.940895 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:35.940863    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7376f235-af7e-4848-9c3e-44c0a1d56714" path="/var/lib/kubelet/pods/7376f235-af7e-4848-9c3e-44c0a1d56714/volumes"
Jul 29 00:48:35.941273 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:35.941258    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6d6b9c1-d71a-4e77-810c-1617370792b2" path="/var/lib/kubelet/pods/e6d6b9c1-d71a-4e77-810c-1617370792b2/volumes"
Jul 29 00:48:56.760085 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:56.760049    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"]
Jul 29 00:48:56.761016 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:56.760966    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq" podUID="5fd1ef26-d466-4c85-ad3a-8e60cea84e29" containerName="maas-api" containerID="cri-o://197f5d8304459a56d9bb92fde3c8ed709de2e20c89417f85c5e5e8fb06ff768c" gracePeriod=30
Jul 29 00:48:56.967449 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:56.967309    2557 generic.go:356] "Generic (PLEG): container finished" podID="5fd1ef26-d466-4c85-ad3a-8e60cea84e29" containerID="197f5d8304459a56d9bb92fde3c8ed709de2e20c89417f85c5e5e8fb06ff768c" exitCode=0
Jul 29 00:48:56.967449 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:56.967392    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq" event={"ID":"5fd1ef26-d466-4c85-ad3a-8e60cea84e29","Type":"ContainerDied","Data":"197f5d8304459a56d9bb92fde3c8ed709de2e20c89417f85c5e5e8fb06ff768c"}
Jul 29 00:48:57.018885 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.018825    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:48:57.109330 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.109301    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-openshift-service-ca\") pod \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") "
Jul 29 00:48:57.109466 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.109341    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqppg\" (UniqueName: \"kubernetes.io/projected/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-kube-api-access-pqppg\") pod \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") "
Jul 29 00:48:57.109466 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.109367    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-maas-api-tls\") pod \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\" (UID: \"5fd1ef26-d466-4c85-ad3a-8e60cea84e29\") "
Jul 29 00:48:57.109711 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.109691    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5fd1ef26-d466-4c85-ad3a-8e60cea84e29" (UID: "5fd1ef26-d466-4c85-ad3a-8e60cea84e29"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:48:57.111526 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.111490    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-kube-api-access-pqppg" (OuterVolumeSpecName: "kube-api-access-pqppg") pod "5fd1ef26-d466-4c85-ad3a-8e60cea84e29" (UID: "5fd1ef26-d466-4c85-ad3a-8e60cea84e29"). InnerVolumeSpecName "kube-api-access-pqppg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:48:57.111526 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.111509    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5fd1ef26-d466-4c85-ad3a-8e60cea84e29" (UID: "5fd1ef26-d466-4c85-ad3a-8e60cea84e29"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:48:57.210807 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.210762    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:48:57.210807 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.210799    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqppg\" (UniqueName: \"kubernetes.io/projected/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-kube-api-access-pqppg\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:48:57.210807 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.210810    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fd1ef26-d466-4c85-ad3a-8e60cea84e29-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 00:48:57.972605 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.972571    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq" event={"ID":"5fd1ef26-d466-4c85-ad3a-8e60cea84e29","Type":"ContainerDied","Data":"900ca0eb0d2aa992b40b6f5460890e21ca1f138ce2faeaa3272c10a70db36d60"}
Jul 29 00:48:57.973039 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.972628    2557 scope.go:117] "RemoveContainer" containerID="197f5d8304459a56d9bb92fde3c8ed709de2e20c89417f85c5e5e8fb06ff768c"
Jul 29 00:48:57.973039 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.972585    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"
Jul 29 00:48:57.986440 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.986411    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"]
Jul 29 00:48:57.989710 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:57.989689    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-111f37-7cfc8d44c8-s54xq"]
Jul 29 00:48:59.943978 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:48:59.943948    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fd1ef26-d466-4c85-ad3a-8e60cea84e29" path="/var/lib/kubelet/pods/5fd1ef26-d466-4c85-ad3a-8e60cea84e29/volumes"
Jul 29 00:49:07.355327 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.355295    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"]
Jul 29 00:49:07.355787 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.355641    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fd1ef26-d466-4c85-ad3a-8e60cea84e29" containerName="maas-api"
Jul 29 00:49:07.355787 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.355652    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fd1ef26-d466-4c85-ad3a-8e60cea84e29" containerName="maas-api"
Jul 29 00:49:07.355787 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.355689    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d6b9c1-d71a-4e77-810c-1617370792b2" containerName="payload-processing"
Jul 29 00:49:07.355787 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.355698    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d6b9c1-d71a-4e77-810c-1617370792b2" containerName="payload-processing"
Jul 29 00:49:07.355787 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.355720    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7376f235-af7e-4848-9c3e-44c0a1d56714" containerName="maas-api"
Jul 29 00:49:07.355787 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.355726    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7376f235-af7e-4848-9c3e-44c0a1d56714" containerName="maas-api"
Jul 29 00:49:07.355787 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.355791    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6d6b9c1-d71a-4e77-810c-1617370792b2" containerName="payload-processing"
Jul 29 00:49:07.356054 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.355801    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7376f235-af7e-4848-9c3e-44c0a1d56714" containerName="maas-api"
Jul 29 00:49:07.356054 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.355808    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fd1ef26-d466-4c85-ad3a-8e60cea84e29" containerName="maas-api"
Jul 29 00:49:07.360360 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.360338    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:07.361872 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.361848    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d90846-serving-cert\""
Jul 29 00:49:07.370291 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.370267    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"]
Jul 29 00:49:07.400866 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.400839    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26b4b369-d1bd-4221-bb82-e81540f24076-maas-api-tls\") pod \"maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:07.401003 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.400884    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b4b369-d1bd-4221-bb82-e81540f24076-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:07.401003 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.400986    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfzdn\" (UniqueName: \"kubernetes.io/projected/26b4b369-d1bd-4221-bb82-e81540f24076-kube-api-access-mfzdn\") pod \"maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:07.501980 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.501946    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26b4b369-d1bd-4221-bb82-e81540f24076-maas-api-tls\") pod \"maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:07.502193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.502005    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b4b369-d1bd-4221-bb82-e81540f24076-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:07.502193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.502061    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfzdn\" (UniqueName: \"kubernetes.io/projected/26b4b369-d1bd-4221-bb82-e81540f24076-kube-api-access-mfzdn\") pod \"maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:07.502193 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:49:07.502133    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-serving-cert: secret "maas-api-e2e-shared-a-d90846-serving-cert" not found
Jul 29 00:49:07.502325 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:49:07.502230    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b4b369-d1bd-4221-bb82-e81540f24076-maas-api-tls podName:26b4b369-d1bd-4221-bb82-e81540f24076 nodeName:}" failed. No retries permitted until 2026-07-29 00:49:08.002206475 +0000 UTC m=+2462.580903912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/26b4b369-d1bd-4221-bb82-e81540f24076-maas-api-tls") pod "maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx" (UID: "26b4b369-d1bd-4221-bb82-e81540f24076") : secret "maas-api-e2e-shared-a-d90846-serving-cert" not found
Jul 29 00:49:07.502761 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.502741    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b4b369-d1bd-4221-bb82-e81540f24076-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:07.528939 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:07.528909    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfzdn\" (UniqueName: \"kubernetes.io/projected/26b4b369-d1bd-4221-bb82-e81540f24076-kube-api-access-mfzdn\") pod \"maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:08.006470 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.006437    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26b4b369-d1bd-4221-bb82-e81540f24076-maas-api-tls\") pod \"maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:08.009083 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.009060    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26b4b369-d1bd-4221-bb82-e81540f24076-maas-api-tls\") pod \"maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:08.151345 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.151302    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"]
Jul 29 00:49:08.154908 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.154887    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 00:49:08.157634 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.157613    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-d90846\""
Jul 29 00:49:08.157789 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.157768    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-d90846-dockercfg-ptvvx\""
Jul 29 00:49:08.166385 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.166360    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"]
Jul 29 00:49:08.209097 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.209059    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5de5300b-7220-495a-b7b3-96cc289d399d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8\" (UID: \"5de5300b-7220-495a-b7b3-96cc289d399d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 00:49:08.209293 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.209114    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjbk4\" (UniqueName: \"kubernetes.io/projected/5de5300b-7220-495a-b7b3-96cc289d399d-kube-api-access-vjbk4\") pod \"payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8\" (UID: \"5de5300b-7220-495a-b7b3-96cc289d399d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 00:49:08.272039 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.271956    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:08.310273 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.310241    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5de5300b-7220-495a-b7b3-96cc289d399d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8\" (UID: \"5de5300b-7220-495a-b7b3-96cc289d399d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 00:49:08.310273 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.310281    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjbk4\" (UniqueName: \"kubernetes.io/projected/5de5300b-7220-495a-b7b3-96cc289d399d-kube-api-access-vjbk4\") pod \"payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8\" (UID: \"5de5300b-7220-495a-b7b3-96cc289d399d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 00:49:08.310929 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.310909    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5de5300b-7220-495a-b7b3-96cc289d399d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8\" (UID: \"5de5300b-7220-495a-b7b3-96cc289d399d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 00:49:08.319511 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.319487    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjbk4\" (UniqueName: \"kubernetes.io/projected/5de5300b-7220-495a-b7b3-96cc289d399d-kube-api-access-vjbk4\") pod \"payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8\" (UID: \"5de5300b-7220-495a-b7b3-96cc289d399d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 00:49:08.467458 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.467426    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 00:49:08.605027 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.605001    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"]
Jul 29 00:49:08.607222 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:49:08.607191    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de5300b_7220_495a_b7b3_96cc289d399d.slice/crio-2248cce62d3a2d26c8a181cc0f75d9098a576598d9c0c81b46b4a67c04e3f2da WatchSource:0}: Error finding container 2248cce62d3a2d26c8a181cc0f75d9098a576598d9c0c81b46b4a67c04e3f2da: Status 404 returned error can't find the container with id 2248cce62d3a2d26c8a181cc0f75d9098a576598d9c0c81b46b4a67c04e3f2da
Jul 29 00:49:08.610887 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:08.610854    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"]
Jul 29 00:49:08.612830 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:49:08.612807    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b4b369_d1bd_4221_bb82_e81540f24076.slice/crio-a0a7b43e4f3d65b97f1ab99d021561aae1150615a3e051f22d98e8a9996c6714 WatchSource:0}: Error finding container a0a7b43e4f3d65b97f1ab99d021561aae1150615a3e051f22d98e8a9996c6714: Status 404 returned error can't find the container with id a0a7b43e4f3d65b97f1ab99d021561aae1150615a3e051f22d98e8a9996c6714
Jul 29 00:49:09.026970 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:09.026934    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8" event={"ID":"5de5300b-7220-495a-b7b3-96cc289d399d","Type":"ContainerStarted","Data":"309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab"}
Jul 29 00:49:09.027156 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:09.026992    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8" event={"ID":"5de5300b-7220-495a-b7b3-96cc289d399d","Type":"ContainerStarted","Data":"2248cce62d3a2d26c8a181cc0f75d9098a576598d9c0c81b46b4a67c04e3f2da"}
Jul 29 00:49:09.027156 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:09.027019    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 00:49:09.028179 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:09.028152    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx" event={"ID":"26b4b369-d1bd-4221-bb82-e81540f24076","Type":"ContainerStarted","Data":"a0a7b43e4f3d65b97f1ab99d021561aae1150615a3e051f22d98e8a9996c6714"}
Jul 29 00:49:09.042186 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:09.042146    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8" podStartSLOduration=1.042132306 podStartE2EDuration="1.042132306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:49:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:49:09.042096797 +0000 UTC m=+2463.620794258" watchObservedRunningTime="2026-07-29 00:49:09.042132306 +0000 UTC m=+2463.620829751"
Jul 29 00:49:10.034243 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:10.034210    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx" event={"ID":"26b4b369-d1bd-4221-bb82-e81540f24076","Type":"ContainerStarted","Data":"934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba"}
Jul 29 00:49:10.050746 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:10.050693    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx" podStartSLOduration=2.36568448 podStartE2EDuration="3.050679084s" totalImagesPullingTime="684.994604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:49:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:49:10.048455884 +0000 UTC m=+2464.627153329" watchObservedRunningTime="2026-07-29 00:49:10.050679084 +0000 UTC m=+2464.629376532"
Jul 29 00:49:11.039035 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:11.039005    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:17.048012 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:17.047985    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 00:49:20.036136 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:20.036105    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 00:49:35.541331 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.541296    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"]
Jul 29 00:49:35.545053 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.545024    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:35.547087 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.547057    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-12505f-serving-cert\""
Jul 29 00:49:35.555822 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.555790    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"]
Jul 29 00:49:35.645561 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.645526    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-maas-api-tls\") pod \"maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:35.645743 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.645635    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szxhs\" (UniqueName: \"kubernetes.io/projected/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-kube-api-access-szxhs\") pod \"maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:35.645743 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.645724    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:35.746934 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.746902    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:35.747101 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.746941    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-maas-api-tls\") pod \"maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:35.747101 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.746994    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szxhs\" (UniqueName: \"kubernetes.io/projected/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-kube-api-access-szxhs\") pod \"maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:35.747181 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:49:35.747114    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-serving-cert: secret "maas-api-e2e-shared-b-12505f-serving-cert" not found
Jul 29 00:49:35.747181 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:49:35.747179    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-maas-api-tls podName:dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a nodeName:}" failed. No retries permitted until 2026-07-29 00:49:36.247160241 +0000 UTC m=+2490.825857665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-maas-api-tls") pod "maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w" (UID: "dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a") : secret "maas-api-e2e-shared-b-12505f-serving-cert" not found
Jul 29 00:49:35.747772 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.747751    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:35.757196 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:35.757169    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szxhs\" (UniqueName: \"kubernetes.io/projected/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-kube-api-access-szxhs\") pod \"maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:36.252822 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:36.252786    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-maas-api-tls\") pod \"maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:36.255226 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:36.255204    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-maas-api-tls\") pod \"maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:36.457844 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:36.457811    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:36.599362 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:36.599320    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"]
Jul 29 00:49:36.602166 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:49:36.602133    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba2fdd0_160a_4bd2_bdbe_d53e1e9b035a.slice/crio-075571e428c32cdab330a67aa0f47c1fa20d5b5b6ec0b11b53f033bb71e62f2a WatchSource:0}: Error finding container 075571e428c32cdab330a67aa0f47c1fa20d5b5b6ec0b11b53f033bb71e62f2a: Status 404 returned error can't find the container with id 075571e428c32cdab330a67aa0f47c1fa20d5b5b6ec0b11b53f033bb71e62f2a
Jul 29 00:49:37.158155 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:37.158125    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w" event={"ID":"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a","Type":"ContainerStarted","Data":"71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58"}
Jul 29 00:49:37.158369 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:37.158334    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:49:37.158437 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:37.158366    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w" event={"ID":"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a","Type":"ContainerStarted","Data":"075571e428c32cdab330a67aa0f47c1fa20d5b5b6ec0b11b53f033bb71e62f2a"}
Jul 29 00:49:37.171875 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:37.171821    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w" podStartSLOduration=1.689233539 podStartE2EDuration="2.171805938s" totalImagesPullingTime="482.572399ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:49:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:49:37.171287175 +0000 UTC m=+2491.749984620" watchObservedRunningTime="2026-07-29 00:49:37.171805938 +0000 UTC m=+2491.750503382"
Jul 29 00:49:43.171412 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:49:43.171380    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 00:53:05.984454 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:53:05.984384    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:53:05.993520 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:53:05.993500    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:58:06.013824 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:58:06.013715    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:58:06.019790 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:58:06.019771    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 00:59:04.603751 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.603670    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"]
Jul 29 00:59:04.607624 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.607598    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:04.613899 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.613876    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-63186d-serving-cert\""
Jul 29 00:59:04.626065 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.626038    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"]
Jul 29 00:59:04.680318 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.680283    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18226c13-d8b7-46af-aa1f-0a4e6c98835f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:04.680487 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.680345    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzfc5\" (UniqueName: \"kubernetes.io/projected/18226c13-d8b7-46af-aa1f-0a4e6c98835f-kube-api-access-wzfc5\") pod \"maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:04.680487 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.680398    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18226c13-d8b7-46af-aa1f-0a4e6c98835f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:04.781339 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.781305    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18226c13-d8b7-46af-aa1f-0a4e6c98835f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:04.781543 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.781374    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18226c13-d8b7-46af-aa1f-0a4e6c98835f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:04.781543 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.781444    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzfc5\" (UniqueName: \"kubernetes.io/projected/18226c13-d8b7-46af-aa1f-0a4e6c98835f-kube-api-access-wzfc5\") pod \"maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:04.781543 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:59:04.781488    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-serving-cert: secret "maas-api-e2e-ipp-a-63186d-serving-cert" not found
Jul 29 00:59:04.781724 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:59:04.781562    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18226c13-d8b7-46af-aa1f-0a4e6c98835f-maas-api-tls podName:18226c13-d8b7-46af-aa1f-0a4e6c98835f nodeName:}" failed. No retries permitted until 2026-07-29 00:59:05.281541802 +0000 UTC m=+3059.860239244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/18226c13-d8b7-46af-aa1f-0a4e6c98835f-maas-api-tls") pod "maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz" (UID: "18226c13-d8b7-46af-aa1f-0a4e6c98835f") : secret "maas-api-e2e-ipp-a-63186d-serving-cert" not found
Jul 29 00:59:04.782090 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.782072    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18226c13-d8b7-46af-aa1f-0a4e6c98835f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:04.792196 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:04.792170    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzfc5\" (UniqueName: \"kubernetes.io/projected/18226c13-d8b7-46af-aa1f-0a4e6c98835f-kube-api-access-wzfc5\") pod \"maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:05.284844 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:05.284806    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18226c13-d8b7-46af-aa1f-0a4e6c98835f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:05.287428 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:05.287402    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18226c13-d8b7-46af-aa1f-0a4e6c98835f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:05.521013 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:05.520973    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:05.658376 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:05.658348    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"]
Jul 29 00:59:05.660770 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:59:05.660732    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18226c13_d8b7_46af_aa1f_0a4e6c98835f.slice/crio-341242a1c85ff5bbd03bd5513185b5eb547b2912cdf26eec09b34fe39e96ae63 WatchSource:0}: Error finding container 341242a1c85ff5bbd03bd5513185b5eb547b2912cdf26eec09b34fe39e96ae63: Status 404 returned error can't find the container with id 341242a1c85ff5bbd03bd5513185b5eb547b2912cdf26eec09b34fe39e96ae63
Jul 29 00:59:05.662193 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:05.662173    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:59:05.747694 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:05.747651    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz" event={"ID":"18226c13-d8b7-46af-aa1f-0a4e6c98835f","Type":"ContainerStarted","Data":"341242a1c85ff5bbd03bd5513185b5eb547b2912cdf26eec09b34fe39e96ae63"}
Jul 29 00:59:06.754044 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:06.753989    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz" event={"ID":"18226c13-d8b7-46af-aa1f-0a4e6c98835f","Type":"ContainerStarted","Data":"4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0"}
Jul 29 00:59:06.754462 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:06.754077    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:06.770456 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:06.770396    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz" podStartSLOduration=2.331921678 podStartE2EDuration="2.770379379s" totalImagesPullingTime="438.457701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:06.769264746 +0000 UTC m=+3061.347962190" watchObservedRunningTime="2026-07-29 00:59:06.770379379 +0000 UTC m=+3061.349076825"
Jul 29 00:59:12.764445 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:12.764414    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 00:59:30.627060 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.627023    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"]
Jul 29 00:59:30.630455 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.630436    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.633271 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.633253    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-efa6ac-openshift-default-dockercfg-sxqcs\""
Jul 29 00:59:30.652839 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.652811    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"]
Jul 29 00:59:30.721930 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.721892    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-credential-socket\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.721930 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.721938    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-envoy\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.722192 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.721961    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-data\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.722192 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.722005    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istiod-ca-cert\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.722192 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.722038    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-certs\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.722192 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.722148    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-token\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.722418 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.722215    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjj7z\" (UniqueName: \"kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-kube-api-access-zjj7z\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.722418 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.722249    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-socket\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.722418 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.722332    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-podinfo\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.822926 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.822873    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-podinfo\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.823131 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.823021    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-credential-socket\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.823131 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.823039    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-envoy\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.823131 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.823056    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-data\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.823131 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.823072    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istiod-ca-cert\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.823131 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.823098    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-certs\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.823131 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.823128    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-token\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.823467 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.823280    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjj7z\" (UniqueName: \"kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-kube-api-access-zjj7z\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.823467 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.823312    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-socket\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.823582 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.823550    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-credential-socket\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.824200 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.824045    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-socket\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.824200 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.824113    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-data\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.824200 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.824118    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-certs\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.824595 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.824567    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istiod-ca-cert\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.825509 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.825484    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-envoy\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.825771 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.825747    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-podinfo\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.830385 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.830356    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-token\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.830797 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.830777    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjj7z\" (UniqueName: \"kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-kube-api-access-zjj7z\") pod \"e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") " pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:30.940604 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:30.940513    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:31.085595 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:31.085561    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"]
Jul 29 00:59:31.088883 ip-10-0-139-127 kubenswrapper[2557]: W0729 00:59:31.088855    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ae6ac64_ef13_444c_96cb_d43a7d64f0c9.slice/crio-536ae70ed16f7acb3d5aef03c52d9e7b991c71ef584126041e51d8a72cad51a5 WatchSource:0}: Error finding container 536ae70ed16f7acb3d5aef03c52d9e7b991c71ef584126041e51d8a72cad51a5: Status 404 returned error can't find the container with id 536ae70ed16f7acb3d5aef03c52d9e7b991c71ef584126041e51d8a72cad51a5
Jul 29 00:59:31.091467 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:31.091428    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:59:31.091530 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:31.091499    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:59:31.091570 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:31.091527    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:59:31.870102 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:31.870068    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" event={"ID":"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9","Type":"ContainerStarted","Data":"ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8"}
Jul 29 00:59:31.870529 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:31.870117    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" event={"ID":"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9","Type":"ContainerStarted","Data":"536ae70ed16f7acb3d5aef03c52d9e7b991c71ef584126041e51d8a72cad51a5"}
Jul 29 00:59:31.887602 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:31.887553    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" podStartSLOduration=1.887536227 podStartE2EDuration="1.887536227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:31.885904964 +0000 UTC m=+3086.464602409" watchObservedRunningTime="2026-07-29 00:59:31.887536227 +0000 UTC m=+3086.466233745"
Jul 29 00:59:31.940935 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:31.940908    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:32.946611 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:32.946581    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:33.879068 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:33.879024    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:33.879843 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:33.879823    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:33.924204 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:33.924172    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"]
Jul 29 00:59:33.928432 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:33.928409    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:33.932071 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:33.931619    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-efa6ac-serving-cert\""
Jul 29 00:59:33.946833 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:33.946808    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"]
Jul 29 00:59:34.055874 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.055844    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.056034 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.055956    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.056034 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.056004    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdqqd\" (UniqueName: \"kubernetes.io/projected/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-kube-api-access-gdqqd\") pod \"maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.156800 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.156713    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.156800 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.156768    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.157024 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.156816    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdqqd\" (UniqueName: \"kubernetes.io/projected/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-kube-api-access-gdqqd\") pod \"maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.157024 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:59:34.156864    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-serving-cert: secret "maas-api-e2e-ipp-b-efa6ac-serving-cert" not found
Jul 29 00:59:34.157024 ip-10-0-139-127 kubenswrapper[2557]: E0729 00:59:34.156924    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-maas-api-tls podName:d0aef1dc-4a0a-4c86-8f96-eed8765937e2 nodeName:}" failed. No retries permitted until 2026-07-29 00:59:34.656907332 +0000 UTC m=+3089.235604755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-maas-api-tls") pod "maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw" (UID: "d0aef1dc-4a0a-4c86-8f96-eed8765937e2") : secret "maas-api-e2e-ipp-b-efa6ac-serving-cert" not found
Jul 29 00:59:34.157428 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.157409    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.165858 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.165828    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdqqd\" (UniqueName: \"kubernetes.io/projected/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-kube-api-access-gdqqd\") pod \"maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.662642 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.662604    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.665072 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.665048    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.846363 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.846321    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:34.992953 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:34.992917    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"]
Jul 29 00:59:35.891039 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:35.891000    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw" event={"ID":"d0aef1dc-4a0a-4c86-8f96-eed8765937e2","Type":"ContainerStarted","Data":"6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10"}
Jul 29 00:59:35.891368 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:35.891055    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw" event={"ID":"d0aef1dc-4a0a-4c86-8f96-eed8765937e2","Type":"ContainerStarted","Data":"3d36d94b3f8d402cb7127f9f38fdbe1397fd576f333337cc36ecc9046a982d8e"}
Jul 29 00:59:35.891368 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:35.891234    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:35.912344 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:35.912298    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw" podStartSLOduration=2.607811767 podStartE2EDuration="2.912282226s" totalImagesPullingTime="304.470459ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:35.910094594 +0000 UTC m=+3090.488792039" watchObservedRunningTime="2026-07-29 00:59:35.912282226 +0000 UTC m=+3090.490979671"
Jul 29 00:59:41.901404 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:41.901375    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 00:59:49.921824 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:49.921783    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 00:59:49.922270 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:49.921854    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" podUID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 00:59:50.965048 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:50.964972    2557 generic.go:356] "Generic (PLEG): container finished" podID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerID="ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8" exitCode=0
Jul 29 00:59:50.965433 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:50.965048    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" event={"ID":"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9","Type":"ContainerDied","Data":"ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8"}
Jul 29 00:59:50.965474 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:50.965434    2557 scope.go:117] "RemoveContainer" containerID="ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8"
Jul 29 00:59:50.966448 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:50.966417    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:59:50.966519 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:50.966487    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:59:50.966555 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:50.966526    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 00:59:51.975905 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:51.975866    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" event={"ID":"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9","Type":"ContainerStarted","Data":"b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5"}
Jul 29 00:59:52.941064 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:52.941029    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:52.941252 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:52.941207    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 29 00:59:52.941305 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:52.941256    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" podUID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 29 00:59:53.941457 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:53.941421    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 29 00:59:53.941866 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:53.941473    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" podUID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 29 00:59:54.941313 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:54.941276    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 29 00:59:54.941496 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:54.941328    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" podUID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 29 00:59:55.944702 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:55.944674    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:55.945075 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:55.945004    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 00:59:55.945725 ip-10-0-139-127 kubenswrapper[2557]: I0729 00:59:55.945708    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 01:00:05.874254 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:05.874220    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-m9sk4_0fb37761-c539-4f63-9dc7-1e94976e2461/payload-processing/0.log"
Jul 29 01:00:53.037807 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:53.037706    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-m9sk4_0fb37761-c539-4f63-9dc7-1e94976e2461/payload-processing/0.log"
Jul 29 01:00:57.304052 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.304018    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"]
Jul 29 01:00:57.308207 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.308184    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:57.310284 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.310259    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-91da6f-serving-cert\""
Jul 29 01:00:57.317322 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.317301    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"]
Jul 29 01:00:57.432936 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.432894    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh2rm\" (UniqueName: \"kubernetes.io/projected/abd5ae4a-8122-4050-9ae7-400a5fe8966b-kube-api-access-rh2rm\") pod \"maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:57.433120 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.432957    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd5ae4a-8122-4050-9ae7-400a5fe8966b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:57.433120 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.433038    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd5ae4a-8122-4050-9ae7-400a5fe8966b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:57.533631 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.533597    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh2rm\" (UniqueName: \"kubernetes.io/projected/abd5ae4a-8122-4050-9ae7-400a5fe8966b-kube-api-access-rh2rm\") pod \"maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:57.533795 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.533643    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd5ae4a-8122-4050-9ae7-400a5fe8966b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:57.533795 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.533711    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd5ae4a-8122-4050-9ae7-400a5fe8966b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:57.533871 ip-10-0-139-127 kubenswrapper[2557]: E0729 01:00:57.533843    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-serving-cert: secret "maas-api-e2e-ipp-cleanup-91da6f-serving-cert" not found
Jul 29 01:00:57.533940 ip-10-0-139-127 kubenswrapper[2557]: E0729 01:00:57.533928    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd5ae4a-8122-4050-9ae7-400a5fe8966b-maas-api-tls podName:abd5ae4a-8122-4050-9ae7-400a5fe8966b nodeName:}" failed. No retries permitted until 2026-07-29 01:00:58.033904601 +0000 UTC m=+3172.612602032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/abd5ae4a-8122-4050-9ae7-400a5fe8966b-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz" (UID: "abd5ae4a-8122-4050-9ae7-400a5fe8966b") : secret "maas-api-e2e-ipp-cleanup-91da6f-serving-cert" not found
Jul 29 01:00:57.534398 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.534373    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd5ae4a-8122-4050-9ae7-400a5fe8966b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:57.541549 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:57.541527    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh2rm\" (UniqueName: \"kubernetes.io/projected/abd5ae4a-8122-4050-9ae7-400a5fe8966b-kube-api-access-rh2rm\") pod \"maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:58.038181 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:58.038139    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd5ae4a-8122-4050-9ae7-400a5fe8966b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:58.041947 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:58.041917    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd5ae4a-8122-4050-9ae7-400a5fe8966b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:58.220676 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:58.220623    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:58.359887 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:58.359790    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"]
Jul 29 01:00:58.362584 ip-10-0-139-127 kubenswrapper[2557]: W0729 01:00:58.362553    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd5ae4a_8122_4050_9ae7_400a5fe8966b.slice/crio-8a7853ba825e544e9241fde22a962bf53b220bdc7f6d21d41b14380a077b0fae WatchSource:0}: Error finding container 8a7853ba825e544e9241fde22a962bf53b220bdc7f6d21d41b14380a077b0fae: Status 404 returned error can't find the container with id 8a7853ba825e544e9241fde22a962bf53b220bdc7f6d21d41b14380a077b0fae
Jul 29 01:00:59.288264 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:59.288228    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz" event={"ID":"abd5ae4a-8122-4050-9ae7-400a5fe8966b","Type":"ContainerStarted","Data":"55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a"}
Jul 29 01:00:59.288487 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:59.288283    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz" event={"ID":"abd5ae4a-8122-4050-9ae7-400a5fe8966b","Type":"ContainerStarted","Data":"8a7853ba825e544e9241fde22a962bf53b220bdc7f6d21d41b14380a077b0fae"}
Jul 29 01:00:59.288487 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:59.288337    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:00:59.304263 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:00:59.304219    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz" podStartSLOduration=1.9012049709999999 podStartE2EDuration="2.304203331s" totalImagesPullingTime="402.99836ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:00:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:00:59.301612343 +0000 UTC m=+3173.880309788" watchObservedRunningTime="2026-07-29 01:00:59.304203331 +0000 UTC m=+3173.882900775"
Jul 29 01:01:05.298808 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:05.298767    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:01:28.015504 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.015466    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"]
Jul 29 01:01:28.015999 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.015713    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz" podUID="abd5ae4a-8122-4050-9ae7-400a5fe8966b" containerName="maas-api" containerID="cri-o://55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a" gracePeriod=30
Jul 29 01:01:28.287530 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.287504    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:01:28.321307 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.321277    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd5ae4a-8122-4050-9ae7-400a5fe8966b-openshift-service-ca\") pod \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") "
Jul 29 01:01:28.321564 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.321445    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh2rm\" (UniqueName: \"kubernetes.io/projected/abd5ae4a-8122-4050-9ae7-400a5fe8966b-kube-api-access-rh2rm\") pod \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") "
Jul 29 01:01:28.321564 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.321478    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd5ae4a-8122-4050-9ae7-400a5fe8966b-maas-api-tls\") pod \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\" (UID: \"abd5ae4a-8122-4050-9ae7-400a5fe8966b\") "
Jul 29 01:01:28.321724 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.321611    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abd5ae4a-8122-4050-9ae7-400a5fe8966b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "abd5ae4a-8122-4050-9ae7-400a5fe8966b" (UID: "abd5ae4a-8122-4050-9ae7-400a5fe8966b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:01:28.321845 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.321822    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abd5ae4a-8122-4050-9ae7-400a5fe8966b-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:01:28.324730 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.324683    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abd5ae4a-8122-4050-9ae7-400a5fe8966b-kube-api-access-rh2rm" (OuterVolumeSpecName: "kube-api-access-rh2rm") pod "abd5ae4a-8122-4050-9ae7-400a5fe8966b" (UID: "abd5ae4a-8122-4050-9ae7-400a5fe8966b"). InnerVolumeSpecName "kube-api-access-rh2rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:01:28.324974 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.324947    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abd5ae4a-8122-4050-9ae7-400a5fe8966b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "abd5ae4a-8122-4050-9ae7-400a5fe8966b" (UID: "abd5ae4a-8122-4050-9ae7-400a5fe8966b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:01:28.423149 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.423123    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh2rm\" (UniqueName: \"kubernetes.io/projected/abd5ae4a-8122-4050-9ae7-400a5fe8966b-kube-api-access-rh2rm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:01:28.423149 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.423148    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abd5ae4a-8122-4050-9ae7-400a5fe8966b-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:01:28.427999 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.427974    2557 generic.go:356] "Generic (PLEG): container finished" podID="abd5ae4a-8122-4050-9ae7-400a5fe8966b" containerID="55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a" exitCode=0
Jul 29 01:01:28.428136 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.428026    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"
Jul 29 01:01:28.428136 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.428048    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz" event={"ID":"abd5ae4a-8122-4050-9ae7-400a5fe8966b","Type":"ContainerDied","Data":"55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a"}
Jul 29 01:01:28.428136 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.428101    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz" event={"ID":"abd5ae4a-8122-4050-9ae7-400a5fe8966b","Type":"ContainerDied","Data":"8a7853ba825e544e9241fde22a962bf53b220bdc7f6d21d41b14380a077b0fae"}
Jul 29 01:01:28.428136 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.428122    2557 scope.go:117] "RemoveContainer" containerID="55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a"
Jul 29 01:01:28.436910 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.436895    2557 scope.go:117] "RemoveContainer" containerID="55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a"
Jul 29 01:01:28.437160 ip-10-0-139-127 kubenswrapper[2557]: E0729 01:01:28.437142    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a\": container with ID starting with 55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a not found: ID does not exist" containerID="55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a"
Jul 29 01:01:28.437218 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.437175    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a"} err="failed to get container status \"55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a\": rpc error: code = NotFound desc = could not find container \"55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a\": container with ID starting with 55d0c736c9af4c31fac5cf919520d727ee06548a88ca8ec3ddf6ecef410c820a not found: ID does not exist"
Jul 29 01:01:28.447479 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.447458    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"]
Jul 29 01:01:28.450853 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:28.450834    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-91da6f-7584b946f5-9cwpz"]
Jul 29 01:01:29.940861 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:29.940830    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abd5ae4a-8122-4050-9ae7-400a5fe8966b" path="/var/lib/kubelet/pods/abd5ae4a-8122-4050-9ae7-400a5fe8966b/volumes"
Jul 29 01:01:53.759614 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:53.759578    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"]
Jul 29 01:01:53.760168 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:53.759887    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz" podUID="18226c13-d8b7-46af-aa1f-0a4e6c98835f" containerName="maas-api" containerID="cri-o://4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0" gracePeriod=30
Jul 29 01:01:54.035826 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.035802    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 01:01:54.160439 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.160409    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18226c13-d8b7-46af-aa1f-0a4e6c98835f-maas-api-tls\") pod \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") "
Jul 29 01:01:54.160603 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.160465    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzfc5\" (UniqueName: \"kubernetes.io/projected/18226c13-d8b7-46af-aa1f-0a4e6c98835f-kube-api-access-wzfc5\") pod \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") "
Jul 29 01:01:54.160603 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.160512    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18226c13-d8b7-46af-aa1f-0a4e6c98835f-openshift-service-ca\") pod \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\" (UID: \"18226c13-d8b7-46af-aa1f-0a4e6c98835f\") "
Jul 29 01:01:54.160918 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.160890    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18226c13-d8b7-46af-aa1f-0a4e6c98835f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "18226c13-d8b7-46af-aa1f-0a4e6c98835f" (UID: "18226c13-d8b7-46af-aa1f-0a4e6c98835f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:01:54.162599 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.162574    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18226c13-d8b7-46af-aa1f-0a4e6c98835f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "18226c13-d8b7-46af-aa1f-0a4e6c98835f" (UID: "18226c13-d8b7-46af-aa1f-0a4e6c98835f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:01:54.162750 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.162680    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18226c13-d8b7-46af-aa1f-0a4e6c98835f-kube-api-access-wzfc5" (OuterVolumeSpecName: "kube-api-access-wzfc5") pod "18226c13-d8b7-46af-aa1f-0a4e6c98835f" (UID: "18226c13-d8b7-46af-aa1f-0a4e6c98835f"). InnerVolumeSpecName "kube-api-access-wzfc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:01:54.261722 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.261690    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18226c13-d8b7-46af-aa1f-0a4e6c98835f-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:01:54.261722 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.261719    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18226c13-d8b7-46af-aa1f-0a4e6c98835f-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:01:54.261912 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.261732    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzfc5\" (UniqueName: \"kubernetes.io/projected/18226c13-d8b7-46af-aa1f-0a4e6c98835f-kube-api-access-wzfc5\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:01:54.549676 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.549611    2557 generic.go:356] "Generic (PLEG): container finished" podID="18226c13-d8b7-46af-aa1f-0a4e6c98835f" containerID="4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0" exitCode=0
Jul 29 01:01:54.549857 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.549683    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"
Jul 29 01:01:54.549857 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.549695    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz" event={"ID":"18226c13-d8b7-46af-aa1f-0a4e6c98835f","Type":"ContainerDied","Data":"4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0"}
Jul 29 01:01:54.549857 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.549760    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz" event={"ID":"18226c13-d8b7-46af-aa1f-0a4e6c98835f","Type":"ContainerDied","Data":"341242a1c85ff5bbd03bd5513185b5eb547b2912cdf26eec09b34fe39e96ae63"}
Jul 29 01:01:54.549857 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.549789    2557 scope.go:117] "RemoveContainer" containerID="4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0"
Jul 29 01:01:54.558784 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.558768    2557 scope.go:117] "RemoveContainer" containerID="4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0"
Jul 29 01:01:54.559029 ip-10-0-139-127 kubenswrapper[2557]: E0729 01:01:54.559011    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0\": container with ID starting with 4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0 not found: ID does not exist" containerID="4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0"
Jul 29 01:01:54.559070 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.559037    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0"} err="failed to get container status \"4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0\": rpc error: code = NotFound desc = could not find container \"4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0\": container with ID starting with 4b6c1ba1b1914eab0b6f8ae33ee1c8814a4d0a92c5e4b3a2ba5c7942db183ab0 not found: ID does not exist"
Jul 29 01:01:54.569642 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.569624    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"]
Jul 29 01:01:54.576247 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:54.576223    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-63186d-d588c6c55-hmndz"]
Jul 29 01:01:55.942183 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:01:55.942151    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18226c13-d8b7-46af-aa1f-0a4e6c98835f" path="/var/lib/kubelet/pods/18226c13-d8b7-46af-aa1f-0a4e6c98835f/volumes"
Jul 29 01:02:53.074426 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.074368    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"]
Jul 29 01:02:53.075492 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.075457    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw" podUID="d0aef1dc-4a0a-4c86-8f96-eed8765937e2" containerName="maas-api" containerID="cri-o://6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10" gracePeriod=30
Jul 29 01:02:53.365603 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.365576    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 01:02:53.395429 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.395404    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-maas-api-tls\") pod \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") "
Jul 29 01:02:53.395589 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.395440    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdqqd\" (UniqueName: \"kubernetes.io/projected/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-kube-api-access-gdqqd\") pod \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") "
Jul 29 01:02:53.395589 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.395460    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-openshift-service-ca\") pod \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\" (UID: \"d0aef1dc-4a0a-4c86-8f96-eed8765937e2\") "
Jul 29 01:02:53.395917 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.395886    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0aef1dc-4a0a-4c86-8f96-eed8765937e2" (UID: "d0aef1dc-4a0a-4c86-8f96-eed8765937e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:02:53.398030 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.397995    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0aef1dc-4a0a-4c86-8f96-eed8765937e2" (UID: "d0aef1dc-4a0a-4c86-8f96-eed8765937e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:02:53.398156 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.398106    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-kube-api-access-gdqqd" (OuterVolumeSpecName: "kube-api-access-gdqqd") pod "d0aef1dc-4a0a-4c86-8f96-eed8765937e2" (UID: "d0aef1dc-4a0a-4c86-8f96-eed8765937e2"). InnerVolumeSpecName "kube-api-access-gdqqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:02:53.496888 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.496849    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:02:53.496888 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.496883    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdqqd\" (UniqueName: \"kubernetes.io/projected/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-kube-api-access-gdqqd\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:02:53.497093 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.496898    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0aef1dc-4a0a-4c86-8f96-eed8765937e2-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:02:53.827156 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.827118    2557 generic.go:356] "Generic (PLEG): container finished" podID="d0aef1dc-4a0a-4c86-8f96-eed8765937e2" containerID="6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10" exitCode=0
Jul 29 01:02:53.827354 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.827167    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"
Jul 29 01:02:53.827354 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.827167    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw" event={"ID":"d0aef1dc-4a0a-4c86-8f96-eed8765937e2","Type":"ContainerDied","Data":"6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10"}
Jul 29 01:02:53.827354 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.827263    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw" event={"ID":"d0aef1dc-4a0a-4c86-8f96-eed8765937e2","Type":"ContainerDied","Data":"3d36d94b3f8d402cb7127f9f38fdbe1397fd576f333337cc36ecc9046a982d8e"}
Jul 29 01:02:53.827354 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.827294    2557 scope.go:117] "RemoveContainer" containerID="6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10"
Jul 29 01:02:53.836876 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.836859    2557 scope.go:117] "RemoveContainer" containerID="6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10"
Jul 29 01:02:53.837137 ip-10-0-139-127 kubenswrapper[2557]: E0729 01:02:53.837118    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10\": container with ID starting with 6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10 not found: ID does not exist" containerID="6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10"
Jul 29 01:02:53.837213 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.837144    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10"} err="failed to get container status \"6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10\": rpc error: code = NotFound desc = could not find container \"6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10\": container with ID starting with 6ff86455d9e510fb707e03acd60c5c1616aa0c285d4d7b2ee938ac0c2030cd10 not found: ID does not exist"
Jul 29 01:02:53.847581 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.847555    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"]
Jul 29 01:02:53.849335 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.849312    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-efa6ac-55f8748c88-4s6lw"]
Jul 29 01:02:53.941313 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:02:53.941270    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0aef1dc-4a0a-4c86-8f96-eed8765937e2" path="/var/lib/kubelet/pods/d0aef1dc-4a0a-4c86-8f96-eed8765937e2/volumes"
Jul 29 01:03:00.288132 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:00.288100    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"]
Jul 29 01:03:00.288499 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:00.288333    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" podUID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerName="istio-proxy" containerID="cri-o://b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5" gracePeriod=30
Jul 29 01:03:05.543393 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.543367    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 01:03:05.599363 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599339    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjj7z\" (UniqueName: \"kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-kube-api-access-zjj7z\") pod \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") "
Jul 29 01:03:05.599526 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599388    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-credential-socket\") pod \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") "
Jul 29 01:03:05.599526 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599403    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-certs\") pod \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") "
Jul 29 01:03:05.599526 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599423    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-socket\") pod \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") "
Jul 29 01:03:05.599526 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599441    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-podinfo\") pod \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") "
Jul 29 01:03:05.599526 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599460    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-data\") pod \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") "
Jul 29 01:03:05.599526 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599479    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-envoy\") pod \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") "
Jul 29 01:03:05.599526 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599507    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istiod-ca-cert\") pod \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") "
Jul 29 01:03:05.599945 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599580    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-token\") pod \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\" (UID: \"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9\") "
Jul 29 01:03:05.599945 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599699    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" (UID: "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:03:05.599945 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599717    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" (UID: "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:03:05.599945 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599779    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" (UID: "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:03:05.600151 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.599991    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-credential-socket\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:05.600151 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.600011    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:05.600151 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.600027    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-workload-socket\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:05.600151 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.600023    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" (UID: "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:03:05.600618 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.600588    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" (UID: "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:03:05.601809 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.601780    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-kube-api-access-zjj7z" (OuterVolumeSpecName: "kube-api-access-zjj7z") pod "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" (UID: "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9"). InnerVolumeSpecName "kube-api-access-zjj7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:03:05.601809 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.601798    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" (UID: "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:03:05.601809 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.601792    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" (UID: "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:03:05.601981 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.601794    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" (UID: "2ae6ac64-ef13-444c-96cb-d43a7d64f0c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 01:03:05.701208 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.701180    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjj7z\" (UniqueName: \"kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-kube-api-access-zjj7z\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:05.701208 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.701206    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-podinfo\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:05.701357 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.701216    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-data\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:05.701357 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.701224    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-envoy\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:05.701357 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.701233    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istiod-ca-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:05.701357 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.701241    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9-istio-token\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:05.889360 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.889277    2557 generic.go:356] "Generic (PLEG): container finished" podID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerID="b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5" exitCode=0
Jul 29 01:03:05.889360 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.889325    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"
Jul 29 01:03:05.889542 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.889369    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" event={"ID":"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9","Type":"ContainerDied","Data":"b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5"}
Jul 29 01:03:05.889542 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.889447    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx" event={"ID":"2ae6ac64-ef13-444c-96cb-d43a7d64f0c9","Type":"ContainerDied","Data":"536ae70ed16f7acb3d5aef03c52d9e7b991c71ef584126041e51d8a72cad51a5"}
Jul 29 01:03:05.889542 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.889471    2557 scope.go:117] "RemoveContainer" containerID="b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5"
Jul 29 01:03:05.898323 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.898308    2557 scope.go:117] "RemoveContainer" containerID="ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8"
Jul 29 01:03:05.905880 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.905866    2557 scope.go:117] "RemoveContainer" containerID="b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5"
Jul 29 01:03:05.906121 ip-10-0-139-127 kubenswrapper[2557]: E0729 01:03:05.906104    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5\": container with ID starting with b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5 not found: ID does not exist" containerID="b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5"
Jul 29 01:03:05.906163 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.906130    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5"} err="failed to get container status \"b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5\": rpc error: code = NotFound desc = could not find container \"b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5\": container with ID starting with b7360b31c10cf393536f08fb482b2603dd9cb3860ca1ab4c4f78a156093673a5 not found: ID does not exist"
Jul 29 01:03:05.906163 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.906146    2557 scope.go:117] "RemoveContainer" containerID="ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8"
Jul 29 01:03:05.906358 ip-10-0-139-127 kubenswrapper[2557]: E0729 01:03:05.906341    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8\": container with ID starting with ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8 not found: ID does not exist" containerID="ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8"
Jul 29 01:03:05.906406 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.906364    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8"} err="failed to get container status \"ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8\": rpc error: code = NotFound desc = could not find container \"ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8\": container with ID starting with ffff17f767d02442fde41268c9bc0c84b60ba8a1cd95f61a176517b88f2d21d8 not found: ID does not exist"
Jul 29 01:03:05.909912 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.909888    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"]
Jul 29 01:03:05.913407 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.913388    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-efa6ac-openshift-default-f9449db48-t6zzx"]
Jul 29 01:03:05.941351 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:05.941325    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" path="/var/lib/kubelet/pods/2ae6ac64-ef13-444c-96cb-d43a7d64f0c9/volumes"
Jul 29 01:03:06.040172 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:06.040144    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 01:03:06.047854 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:06.047833    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77kdj_eda34408-bf29-4613-acd9-b6db5f690d9f/ovn-acl-logging/0.log"
Jul 29 01:03:11.103239 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.103206    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"]
Jul 29 01:03:11.103690 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.103441    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx" podUID="26b4b369-d1bd-4221-bb82-e81540f24076" containerName="maas-api" containerID="cri-o://934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba" gracePeriod=30
Jul 29 01:03:11.160353 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.160317    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"]
Jul 29 01:03:11.160574 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.160541    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8" podUID="5de5300b-7220-495a-b7b3-96cc289d399d" containerName="payload-pre-processing" containerID="cri-o://309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab" gracePeriod=30
Jul 29 01:03:11.404373 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.404346    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 01:03:11.440705 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.440679    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 01:03:11.451523 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.451502    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b4b369-d1bd-4221-bb82-e81540f24076-openshift-service-ca\") pod \"26b4b369-d1bd-4221-bb82-e81540f24076\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") "
Jul 29 01:03:11.451668 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.451559    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfzdn\" (UniqueName: \"kubernetes.io/projected/26b4b369-d1bd-4221-bb82-e81540f24076-kube-api-access-mfzdn\") pod \"26b4b369-d1bd-4221-bb82-e81540f24076\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") "
Jul 29 01:03:11.451668 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.451610    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26b4b369-d1bd-4221-bb82-e81540f24076-maas-api-tls\") pod \"26b4b369-d1bd-4221-bb82-e81540f24076\" (UID: \"26b4b369-d1bd-4221-bb82-e81540f24076\") "
Jul 29 01:03:11.451908 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.451880    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26b4b369-d1bd-4221-bb82-e81540f24076-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26b4b369-d1bd-4221-bb82-e81540f24076" (UID: "26b4b369-d1bd-4221-bb82-e81540f24076"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:03:11.454262 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.454234    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26b4b369-d1bd-4221-bb82-e81540f24076-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26b4b369-d1bd-4221-bb82-e81540f24076" (UID: "26b4b369-d1bd-4221-bb82-e81540f24076"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:03:11.454367 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.454237    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26b4b369-d1bd-4221-bb82-e81540f24076-kube-api-access-mfzdn" (OuterVolumeSpecName: "kube-api-access-mfzdn") pod "26b4b369-d1bd-4221-bb82-e81540f24076" (UID: "26b4b369-d1bd-4221-bb82-e81540f24076"). InnerVolumeSpecName "kube-api-access-mfzdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:03:11.552401 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.552365    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5de5300b-7220-495a-b7b3-96cc289d399d-plugins-config-volume\") pod \"5de5300b-7220-495a-b7b3-96cc289d399d\" (UID: \"5de5300b-7220-495a-b7b3-96cc289d399d\") "
Jul 29 01:03:11.552571 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.552412    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjbk4\" (UniqueName: \"kubernetes.io/projected/5de5300b-7220-495a-b7b3-96cc289d399d-kube-api-access-vjbk4\") pod \"5de5300b-7220-495a-b7b3-96cc289d399d\" (UID: \"5de5300b-7220-495a-b7b3-96cc289d399d\") "
Jul 29 01:03:11.552710 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.552683    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b4b369-d1bd-4221-bb82-e81540f24076-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:11.552710 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.552708    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfzdn\" (UniqueName: \"kubernetes.io/projected/26b4b369-d1bd-4221-bb82-e81540f24076-kube-api-access-mfzdn\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:11.552869 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.552724    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26b4b369-d1bd-4221-bb82-e81540f24076-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:11.552869 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.552802    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5de5300b-7220-495a-b7b3-96cc289d399d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5de5300b-7220-495a-b7b3-96cc289d399d" (UID: "5de5300b-7220-495a-b7b3-96cc289d399d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:03:11.554450 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.554429    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5de5300b-7220-495a-b7b3-96cc289d399d-kube-api-access-vjbk4" (OuterVolumeSpecName: "kube-api-access-vjbk4") pod "5de5300b-7220-495a-b7b3-96cc289d399d" (UID: "5de5300b-7220-495a-b7b3-96cc289d399d"). InnerVolumeSpecName "kube-api-access-vjbk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:03:11.654145 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.654074    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5de5300b-7220-495a-b7b3-96cc289d399d-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:11.654145 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.654100    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjbk4\" (UniqueName: \"kubernetes.io/projected/5de5300b-7220-495a-b7b3-96cc289d399d-kube-api-access-vjbk4\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:11.917192 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.917100    2557 generic.go:356] "Generic (PLEG): container finished" podID="5de5300b-7220-495a-b7b3-96cc289d399d" containerID="309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab" exitCode=0
Jul 29 01:03:11.917192 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.917151    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"
Jul 29 01:03:11.917192 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.917181    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8" event={"ID":"5de5300b-7220-495a-b7b3-96cc289d399d","Type":"ContainerDied","Data":"309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab"}
Jul 29 01:03:11.917452 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.917235    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8" event={"ID":"5de5300b-7220-495a-b7b3-96cc289d399d","Type":"ContainerDied","Data":"2248cce62d3a2d26c8a181cc0f75d9098a576598d9c0c81b46b4a67c04e3f2da"}
Jul 29 01:03:11.917452 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.917265    2557 scope.go:117] "RemoveContainer" containerID="309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab"
Jul 29 01:03:11.918565 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.918544    2557 generic.go:356] "Generic (PLEG): container finished" podID="26b4b369-d1bd-4221-bb82-e81540f24076" containerID="934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba" exitCode=0
Jul 29 01:03:11.918643 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.918583    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"
Jul 29 01:03:11.918735 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.918629    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx" event={"ID":"26b4b369-d1bd-4221-bb82-e81540f24076","Type":"ContainerDied","Data":"934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba"}
Jul 29 01:03:11.918735 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.918707    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx" event={"ID":"26b4b369-d1bd-4221-bb82-e81540f24076","Type":"ContainerDied","Data":"a0a7b43e4f3d65b97f1ab99d021561aae1150615a3e051f22d98e8a9996c6714"}
Jul 29 01:03:11.929298 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.929083    2557 scope.go:117] "RemoveContainer" containerID="309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab"
Jul 29 01:03:11.929596 ip-10-0-139-127 kubenswrapper[2557]: E0729 01:03:11.929566    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab\": container with ID starting with 309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab not found: ID does not exist" containerID="309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab"
Jul 29 01:03:11.929739 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.929605    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab"} err="failed to get container status \"309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab\": rpc error: code = NotFound desc = could not find container \"309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab\": container with ID starting with 309390b85c518bc142d8e397573db3070344b47b8e59ec000efa7023e43533ab not found: ID does not exist"
Jul 29 01:03:11.929739 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.929628    2557 scope.go:117] "RemoveContainer" containerID="934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba"
Jul 29 01:03:11.937708 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.937692    2557 scope.go:117] "RemoveContainer" containerID="934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba"
Jul 29 01:03:11.937944 ip-10-0-139-127 kubenswrapper[2557]: E0729 01:03:11.937921    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba\": container with ID starting with 934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba not found: ID does not exist" containerID="934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba"
Jul 29 01:03:11.938014 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.937955    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba"} err="failed to get container status \"934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba\": rpc error: code = NotFound desc = could not find container \"934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba\": container with ID starting with 934f12137e32da9eb04056c9e5481b7cbc734961e9835effdafb23e29a9917ba not found: ID does not exist"
Jul 29 01:03:11.942507 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.942488    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"]
Jul 29 01:03:11.944941 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.944921    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-d90846-5ff4fc795b-tftq8"]
Jul 29 01:03:11.954088 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.954051    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"]
Jul 29 01:03:11.956666 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:11.956636    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d90846-8c46d848b-bjhtx"]
Jul 29 01:03:13.940160 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:13.940129    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26b4b369-d1bd-4221-bb82-e81540f24076" path="/var/lib/kubelet/pods/26b4b369-d1bd-4221-bb82-e81540f24076/volumes"
Jul 29 01:03:13.940540 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:13.940476    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5de5300b-7220-495a-b7b3-96cc289d399d" path="/var/lib/kubelet/pods/5de5300b-7220-495a-b7b3-96cc289d399d/volumes"
Jul 29 01:03:28.522910 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:28.522681    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"]
Jul 29 01:03:28.523409 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:28.522968    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w" podUID="dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a" containerName="maas-api" containerID="cri-o://71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58" gracePeriod=30
Jul 29 01:03:28.781573 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:28.781513    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 01:03:28.905936 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:28.905896    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szxhs\" (UniqueName: \"kubernetes.io/projected/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-kube-api-access-szxhs\") pod \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") "
Jul 29 01:03:28.906129 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:28.905957    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-maas-api-tls\") pod \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") "
Jul 29 01:03:28.906129 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:28.905983    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-openshift-service-ca\") pod \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\" (UID: \"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a\") "
Jul 29 01:03:28.906416 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:28.906382    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a" (UID: "dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:03:28.908054 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:28.908022    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a" (UID: "dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:03:28.908162 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:28.908109    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-kube-api-access-szxhs" (OuterVolumeSpecName: "kube-api-access-szxhs") pod "dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a" (UID: "dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a"). InnerVolumeSpecName "kube-api-access-szxhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:03:29.002242 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.002212    2557 generic.go:356] "Generic (PLEG): container finished" podID="dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a" containerID="71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58" exitCode=0
Jul 29 01:03:29.002409 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.002255    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"
Jul 29 01:03:29.002409 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.002307    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w" event={"ID":"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a","Type":"ContainerDied","Data":"71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58"}
Jul 29 01:03:29.002409 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.002362    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w" event={"ID":"dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a","Type":"ContainerDied","Data":"075571e428c32cdab330a67aa0f47c1fa20d5b5b6ec0b11b53f033bb71e62f2a"}
Jul 29 01:03:29.002409 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.002389    2557 scope.go:117] "RemoveContainer" containerID="71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58"
Jul 29 01:03:29.006910 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.006882    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szxhs\" (UniqueName: \"kubernetes.io/projected/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-kube-api-access-szxhs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:29.006910 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.006910    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:29.007107 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.006925    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Jul 29 01:03:29.011371 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.011355    2557 scope.go:117] "RemoveContainer" containerID="71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58"
Jul 29 01:03:29.011629 ip-10-0-139-127 kubenswrapper[2557]: E0729 01:03:29.011612    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58\": container with ID starting with 71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58 not found: ID does not exist" containerID="71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58"
Jul 29 01:03:29.011742 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.011641    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58"} err="failed to get container status \"71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58\": rpc error: code = NotFound desc = could not find container \"71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58\": container with ID starting with 71e682a734d304fc831b943f881e331e5c9e6df6657a83fe2d743a86bbfc7d58 not found: ID does not exist"
Jul 29 01:03:29.024422 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.024397    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"]
Jul 29 01:03:29.029741 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.029720    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-12505f-66cf57f44f-htt7w"]
Jul 29 01:03:29.940707 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:29.940674    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a" path="/var/lib/kubelet/pods/dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a/volumes"
Jul 29 01:03:41.822701 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:41.822646    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-czwkv_bf9b582a-ae98-4d10-82d6-91bef055b4d0/manager/0.log"
Jul 29 01:03:41.943291 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:41.943260    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qx7pl_77256c38-1517-4492-a989-35a7957b1989/manager/0.log"
Jul 29 01:03:42.066580 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:42.066534    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f9bf644fc-tp8r8_96c5fa1d-10b9-4b8d-8af5-3155173f9e44/manager/0.log"
Jul 29 01:03:42.539591 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:42.539566    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fc48f86bc-plwt5_5351c052-35c1-48b9-80a9-487c5d30a401/copy-manifests/0.log"
Jul 29 01:03:42.546556 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:42.546529    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fc48f86bc-plwt5_5351c052-35c1-48b9-80a9-487c5d30a401/manager/0.log"
Jul 29 01:03:43.015845 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:43.015815    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6cf8f75bb9-9d8rd_1e8899a7-b737-435b-a63b-2c1a8b7c1b89/maas-api/0.log"
Jul 29 01:03:43.471828 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:43.471793    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-5ws8n_6147e48b-b472-4a5c-b740-01fbb4ce1e8e/postgres/0.log"
Jul 29 01:03:46.074641 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:46.074613    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q8ckc_35301f60-9663-4cce-ac09-e002b79110ca/discovery/0.log"
Jul 29 01:03:46.291358 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:46.291320    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6998fb56fc-xczc2_0230a81a-283c-42f5-8d91-a7a66aec73d3/kube-auth-proxy/0.log"
Jul 29 01:03:46.418043 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:46.418008    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hfcwr_3070aafb-0177-4490-8ba4-d051480325fd/istio-proxy/1.log"
Jul 29 01:03:46.541979 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:46.541939    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-jtz6s_2a6ffe48-47d6-4470-8d57-1b16766a4cd7/payload-pre-processing/0.log"
Jul 29 01:03:46.664434 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:46.664401    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-m9sk4_0fb37761-c539-4f63-9dc7-1e94976e2461/payload-processing/0.log"
Jul 29 01:03:47.144844 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:47.144817    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-t54g8_0d88483d-5f75-49f7-9120-01f2a8c28e49/main/0.log"
Jul 29 01:03:47.263699 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:47.263671    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6hhwz_1d5c50f4-decb-4192-8d99-9a750e5bc5ed/main/0.log"
Jul 29 01:03:47.603559 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:47.603532    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6rf6r_2b7623c1-67ae-4c29-9405-c7a9bde8ecdd/main/0.log"
Jul 29 01:03:56.342126 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:56.342093    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-njnlq_a611c2ee-2d8f-4099-ad41-d2be9a0497fa/global-pull-secret-syncer/0.log"
Jul 29 01:03:56.475579 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:56.475547    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cjlpv_42f50e14-6d95-4eeb-b70c-e03291a251b3/konnectivity-agent/0.log"
Jul 29 01:03:56.569234 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:03:56.569196    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-127.ec2.internal_aeef17ba4dc68f055aaf49917ef0d775/haproxy/0.log"
Jul 29 01:04:04.472423 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:04.472388    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-phwj2_2366eb7b-7d82-4640-9a65-556275f8b6c0/node-exporter/0.log"
Jul 29 01:04:04.491753 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:04.491730    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-phwj2_2366eb7b-7d82-4640-9a65-556275f8b6c0/kube-rbac-proxy/0.log"
Jul 29 01:04:04.508653 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:04.508631    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-phwj2_2366eb7b-7d82-4640-9a65-556275f8b6c0/init-textfile/0.log"
Jul 29 01:04:06.298918 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.298885    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"]
Jul 29 01:04:06.299258 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299211    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerName="istio-proxy"
Jul 29 01:04:06.299258 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299222    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerName="istio-proxy"
Jul 29 01:04:06.299258 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299235    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5de5300b-7220-495a-b7b3-96cc289d399d" containerName="payload-pre-processing"
Jul 29 01:04:06.299258 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299240    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5de5300b-7220-495a-b7b3-96cc289d399d" containerName="payload-pre-processing"
Jul 29 01:04:06.299258 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299246    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b4b369-d1bd-4221-bb82-e81540f24076" containerName="maas-api"
Jul 29 01:04:06.299258 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299254    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b4b369-d1bd-4221-bb82-e81540f24076" containerName="maas-api"
Jul 29 01:04:06.299258 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299261    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abd5ae4a-8122-4050-9ae7-400a5fe8966b" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299267    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="abd5ae4a-8122-4050-9ae7-400a5fe8966b" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299278    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299283    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299291    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0aef1dc-4a0a-4c86-8f96-eed8765937e2" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299297    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0aef1dc-4a0a-4c86-8f96-eed8765937e2" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299307    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18226c13-d8b7-46af-aa1f-0a4e6c98835f" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299313    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="18226c13-d8b7-46af-aa1f-0a4e6c98835f" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299366    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerName="istio-proxy"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299374    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="26b4b369-d1bd-4221-bb82-e81540f24076" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299381    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0aef1dc-4a0a-4c86-8f96-eed8765937e2" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299386    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="abd5ae4a-8122-4050-9ae7-400a5fe8966b" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299392    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba2fdd0-160a-4bd2-bdbe-d53e1e9b035a" containerName="maas-api"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299397    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ae6ac64-ef13-444c-96cb-d43a7d64f0c9" containerName="istio-proxy"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299402    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5de5300b-7220-495a-b7b3-96cc289d399d" containerName="payload-pre-processing"
Jul 29 01:04:06.299488 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.299408    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="18226c13-d8b7-46af-aa1f-0a4e6c98835f" containerName="maas-api"
Jul 29 01:04:06.303748 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.303728    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.305355 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.305335    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dx6mt\"/\"openshift-service-ca.crt\""
Jul 29 01:04:06.305464 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.305435    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dx6mt\"/\"default-dockercfg-xvggr\""
Jul 29 01:04:06.305512 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.305435    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dx6mt\"/\"kube-root-ca.crt\""
Jul 29 01:04:06.314698 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.314653    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"]
Jul 29 01:04:06.453096 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.453064    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-sys\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.453288 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.453098    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s569\" (UniqueName: \"kubernetes.io/projected/eba95b38-3498-4965-8865-28466b813008-kube-api-access-9s569\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.453288 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.453184    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-podres\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.453288 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.453263    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-lib-modules\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.453449 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.453299    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-proc\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.554373 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.554279    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-lib-modules\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.554373 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.554321    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-proc\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.554595 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.554384    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-sys\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.554595 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.554406    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s569\" (UniqueName: \"kubernetes.io/projected/eba95b38-3498-4965-8865-28466b813008-kube-api-access-9s569\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.554595 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.554452    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-podres\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.554595 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.554470    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-lib-modules\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.554595 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.554481    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-proc\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.554595 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.554499    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-sys\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.554595 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.554562    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eba95b38-3498-4965-8865-28466b813008-podres\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.561149 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.561131    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s569\" (UniqueName: \"kubernetes.io/projected/eba95b38-3498-4965-8865-28466b813008-kube-api-access-9s569\") pod \"perf-node-gather-daemonset-p7tsb\" (UID: \"eba95b38-3498-4965-8865-28466b813008\") " pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.621118 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.621083    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:06.752512 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:06.752485    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"]
Jul 29 01:04:06.757108 ip-10-0-139-127 kubenswrapper[2557]: W0729 01:04:06.757046    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podeba95b38_3498_4965_8865_28466b813008.slice/crio-722d7cb78a70991c340fd049cf0391e8c4076dadb1c7e6faa1bec8cc70770d9b WatchSource:0}: Error finding container 722d7cb78a70991c340fd049cf0391e8c4076dadb1c7e6faa1bec8cc70770d9b: Status 404 returned error can't find the container with id 722d7cb78a70991c340fd049cf0391e8c4076dadb1c7e6faa1bec8cc70770d9b
Jul 29 01:04:07.173075 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:07.173038    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb" event={"ID":"eba95b38-3498-4965-8865-28466b813008","Type":"ContainerStarted","Data":"a37f30f9bdca939c2f9e2eac8fb3b33e5d1b8a750e31899558548f4c518a98dd"}
Jul 29 01:04:07.173256 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:07.173098    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb" event={"ID":"eba95b38-3498-4965-8865-28466b813008","Type":"ContainerStarted","Data":"722d7cb78a70991c340fd049cf0391e8c4076dadb1c7e6faa1bec8cc70770d9b"}
Jul 29 01:04:07.173256 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:07.173138    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:07.187516 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:07.187471    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb" podStartSLOduration=1.187457346 podStartE2EDuration="1.187457346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:04:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:04:07.186150755 +0000 UTC m=+3361.764848201" watchObservedRunningTime="2026-07-29 01:04:07.187457346 +0000 UTC m=+3361.766154798"
Jul 29 01:04:07.352552 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:07.352519    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-55f65b58b5-zc8qq_ffd9a1b7-12b4-4629-827d-8cd4ae7c7f2b/console/0.log"
Jul 29 01:04:09.046828 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:09.046787    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tm4dr_4bff27cb-47c4-412d-8e1f-8de040279666/dns/0.log"
Jul 29 01:04:09.063945 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:09.063920    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tm4dr_4bff27cb-47c4-412d-8e1f-8de040279666/kube-rbac-proxy/0.log"
Jul 29 01:04:09.105111 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:09.105084    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j2hc8_ce867f86-40e1-48ce-9a7d-a3264f1ca054/dns-node-resolver/0.log"
Jul 29 01:04:09.642507 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:09.642476    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-684c6f67d8-sbz2x_5da1ca5a-517b-4f53-b79f-261c58b25a22/registry/0.log"
Jul 29 01:04:09.714876 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:09.714847    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qz47f_7064533e-41f2-46e9-b3ff-a95ae591a5a4/node-ca/0.log"
Jul 29 01:04:10.866102 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:10.866074    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q8ckc_35301f60-9663-4cce-ac09-e002b79110ca/discovery/0.log"
Jul 29 01:04:10.908506 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:10.908473    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6998fb56fc-xczc2_0230a81a-283c-42f5-8d91-a7a66aec73d3/kube-auth-proxy/0.log"
Jul 29 01:04:10.934142 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:10.934113    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hfcwr_3070aafb-0177-4490-8ba4-d051480325fd/istio-proxy/0.log"
Jul 29 01:04:10.992502 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:10.992477    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hfcwr_3070aafb-0177-4490-8ba4-d051480325fd/istio-proxy/1.log"
Jul 29 01:04:11.031163 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:11.031134    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-jtz6s_2a6ffe48-47d6-4470-8d57-1b16766a4cd7/payload-pre-processing/0.log"
Jul 29 01:04:11.067943 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:11.067919    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-m9sk4_0fb37761-c539-4f63-9dc7-1e94976e2461/payload-processing/0.log"
Jul 29 01:04:11.910116 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:11.910084    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ptczj_c9af53b3-10d5-4075-acd0-421f63b08456/serve-healthcheck-canary/0.log"
Jul 29 01:04:12.562086 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:12.562055    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vd2sr_e94e1019-75e2-4403-bfa0-20e6fdc8bd51/kube-rbac-proxy/0.log"
Jul 29 01:04:12.580171 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:12.580149    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vd2sr_e94e1019-75e2-4403-bfa0-20e6fdc8bd51/exporter/0.log"
Jul 29 01:04:12.605667 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:12.605634    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vd2sr_e94e1019-75e2-4403-bfa0-20e6fdc8bd51/extractor/0.log"
Jul 29 01:04:13.186865 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:13.186834    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dx6mt/perf-node-gather-daemonset-p7tsb"
Jul 29 01:04:14.688413 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:14.688378    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-czwkv_bf9b582a-ae98-4d10-82d6-91bef055b4d0/manager/0.log"
Jul 29 01:04:15.080920 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:15.080832    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qx7pl_77256c38-1517-4492-a989-35a7957b1989/manager/0.log"
Jul 29 01:04:15.317489 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:15.317459    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f9bf644fc-tp8r8_96c5fa1d-10b9-4b8d-8af5-3155173f9e44/manager/0.log"
Jul 29 01:04:15.566724 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:15.566692    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fc48f86bc-plwt5_5351c052-35c1-48b9-80a9-487c5d30a401/manager/0.log"
Jul 29 01:04:15.588023 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:15.587999    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fc48f86bc-plwt5_5351c052-35c1-48b9-80a9-487c5d30a401/copy-manifests/0.log"
Jul 29 01:04:16.966032 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:16.965927    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-gswx7_4b67f99f-a7d4-4d65-937d-127e2edcf5e3/openshift-lws-operator/0.log"
Jul 29 01:04:17.206931 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:17.206859    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q8ckc_35301f60-9663-4cce-ac09-e002b79110ca/discovery/0.log"
Jul 29 01:04:17.232094 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:17.232022    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6998fb56fc-xczc2_0230a81a-283c-42f5-8d91-a7a66aec73d3/kube-auth-proxy/0.log"
Jul 29 01:04:17.253077 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:17.253046    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hfcwr_3070aafb-0177-4490-8ba4-d051480325fd/istio-proxy/0.log"
Jul 29 01:04:17.306887 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:17.306854    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hfcwr_3070aafb-0177-4490-8ba4-d051480325fd/istio-proxy/1.log"
Jul 29 01:04:17.341462 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:17.341423    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-jtz6s_2a6ffe48-47d6-4470-8d57-1b16766a4cd7/payload-pre-processing/0.log"
Jul 29 01:04:17.371928 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:17.371901    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-m9sk4_0fb37761-c539-4f63-9dc7-1e94976e2461/payload-processing/0.log"
Jul 29 01:04:18.974445 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:18.974414    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q8ckc_35301f60-9663-4cce-ac09-e002b79110ca/discovery/0.log"
Jul 29 01:04:19.002064 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:19.002038    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6998fb56fc-xczc2_0230a81a-283c-42f5-8d91-a7a66aec73d3/kube-auth-proxy/0.log"
Jul 29 01:04:19.022185 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:19.022157    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hfcwr_3070aafb-0177-4490-8ba4-d051480325fd/istio-proxy/0.log"
Jul 29 01:04:19.084904 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:19.084879    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hfcwr_3070aafb-0177-4490-8ba4-d051480325fd/istio-proxy/1.log"
Jul 29 01:04:19.115821 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:19.115794    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-jtz6s_2a6ffe48-47d6-4470-8d57-1b16766a4cd7/payload-pre-processing/0.log"
Jul 29 01:04:19.145565 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:19.145540    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-m9sk4_0fb37761-c539-4f63-9dc7-1e94976e2461/payload-processing/0.log"
Jul 29 01:04:21.790727 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:21.790697    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-dhmhg_683ea5f6-3cec-4bc4-b979-9ea0385646b8/migrator/0.log"
Jul 29 01:04:21.810079 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:21.810047    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-dhmhg_683ea5f6-3cec-4bc4-b979-9ea0385646b8/graceful-termination/0.log"
Jul 29 01:04:23.152316 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:23.152283    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-85dnm_3201f9ff-9853-4b96-8b6e-0432d625d30c/kube-multus/0.log"
Jul 29 01:04:23.331085 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:23.331058    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tsqzq_42be31c5-c94c-4fcd-982b-d8cd39f4b38c/kube-multus-additional-cni-plugins/0.log"
Jul 29 01:04:23.359353 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:23.359326    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tsqzq_42be31c5-c94c-4fcd-982b-d8cd39f4b38c/egress-router-binary-copy/0.log"
Jul 29 01:04:23.379683 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:23.379649    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tsqzq_42be31c5-c94c-4fcd-982b-d8cd39f4b38c/cni-plugins/0.log"
Jul 29 01:04:23.400211 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:23.400194    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tsqzq_42be31c5-c94c-4fcd-982b-d8cd39f4b38c/bond-cni-plugin/0.log"
Jul 29 01:04:23.420746 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:23.420686    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tsqzq_42be31c5-c94c-4fcd-982b-d8cd39f4b38c/routeoverride-cni/0.log"
Jul 29 01:04:23.443222 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:23.443198    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tsqzq_42be31c5-c94c-4fcd-982b-d8cd39f4b38c/whereabouts-cni-bincopy/0.log"
Jul 29 01:04:23.463616 ip-10-0-139-127 kubenswrapper[2557]: I0729 01:04:23.463592    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tsqzq_42be31c5-c94c-4fcd-982b-d8cd39f4b38c/whereabouts-cni/0.log"